Affiliations
Department of Medicine, University of Chicago
Email
dperes@medicine.bsd.uchicago.edu
Given name(s)
Dana P.
Family name
Edelson
Degrees
MD, MS

Accuracy Comparisons between Manual and Automated Respiratory Rate for Detecting Clinical Deterioration in Ward Patients

Article Type
Changed
Wed, 08/15/2018 - 06:54

Respiratory rate is the most accurate vital sign for predicting adverse outcomes in ward patients.1,2 Though other vital signs are typically collected by using machines, respiratory rate is collected manually by caregivers counting the breathing rate. However, studies have shown significant discrepancies between a patient’s respiratory rate documented in the medical record, which is often 18 or 20, and the value measured by counting the rate over a full minute.3 Thus, despite the high accuracy of respiratory rate, it is possible that these values do not represent true patient physiology. It is unknown whether a valid automated measurement of respiratory rate would be more predictive than a manually collected respiratory rate for identifying patients who develop deterioration. The aim of this study was to compare the distribution and predictive accuracy of manually and automatically recorded respiratory rates.

METHODS

In this prospective cohort study, adult patients admitted to one oncology ward at the University of Chicago from April 2015 to May 2016 were approached for consent (Institutional Review Board #14-0682). Enrolled patients were fit with a cableless, FDA-approved respiratory pod device (Philips IntelliVue clResp Pod; Philips Healthcare, Andover, MA) that automatically recorded respiratory rate and heart rate every 15 minutes while they remained on the ward. Pod data were paired with vital sign data documented in the electronic health record (EHR) by taking the automated value closest, but prior to, the manual value up to a maximum of 4 hours. Automated and manual respiratory rate were compared by using the area under the receiver operating characteristic curve (AUC) for whether an intensive care unit (ICU) transfer occurred within 24 hours of each paired observation without accounting for patient-level clustering.

RESULTS

A total of 1402 paired respiratory rate observations from 51 patient admissions were included, of which 5 patients (9.8%) experienced an ICU transfer. Paired heart rate values were highly correlated (r = 0.86), while paired respiratory rate values were less correlated (r = 0.38). The automated values had a median of 21 (interquartile range [IQR] of 17-25), while the manual values had a median of 18 (IQR of 16-21). Manual respiratory rates were significantly more accurate for predicting ICU transfer than automated respiratory rates (AUC 0.67 [95% CI, 0.62-0.73] vs 0.60 [95% CI, 0.55-0.65]; P = .011). As shown in the Figure, accuracy was similar between manual and automated respiratory rates until 18 breaths per minute, above which the manual respiratory rates were more predictive. At a threshold with similar specificity, manual respiratory rates >22 had a sensitivity of 45% and specificity of 84%, while automated respiratory rates >26 had a sensitivity of 22% and specificity of 81%. At a threshold with similar sensitivity, manual respiratory rates >20 had a sensitivity of 54% and specificity of 75%, while automated respiratory rates >22 had a sensitivity of 52% and specificity of 64%.

DISCUSSION

In this prospective cohort study, we found that manual respiratory rates were different than those collected from an automated system and, yet, were significantly more accurate for predicting ICU transfer. These results suggest that the predictive accuracy of respiratory rates documented in the EHR is due to more than just physiology. Our findings have important implications for the risk stratification of ward patients.

Though previous literature has suggested that respiratory rate is the most accurate predictor of deterioration, this may not be true.1 Respiratory rates manually recorded by clinical staff may contain information beyond pure physiology, such as a proxy of clinician concern, which may inflate the predictive value. Nursing staff may record standard respiratory rate values for patients that appear to be well (eg, 18) but count actual rates for those patients they suspect have a more severe disease, which is one possible explanation for our findings. In addition, automated assessments are likely to be more sensitive to intermittent fluctuations in respiratory rate associated with patient movement or emotion. This might explain the improved accuracy at higher rates for manually recorded vital signs.

Although limited by its small sample size, our results have important implications for patient monitoring and early warning scores designed to identify high-risk ward patients given that both simple scores and statistically derived models include respiratory rates as a predictor.4 As hospitals move to use newer technologies to automate vital sign monitoring and decrease nursing workload, our findings suggest that accuracy for identifying high-risk patients may be lost. Additional methods for capturing subjective assessments from clinical providers may be necessary and could be incorporated into risk scores.5 For example, the 7-point subjective Patient Acuity Rating has been shown to augment the Modified Early Warning Score for predicting ICU transfer, rapid response activation, or cardiac arrest within 24 hours.6

Manually recorded respiratory rate may include information beyond pure physiology, which inflates its predictive value. This has important implications for the use of automated monitoring technology in hospitals and the integration of these measurements into early warning scores.

 

 

Acknowledgments

The authors thank Pamela McCall, BSN, OCN for her assistance with study implementation, Kevin Ig-Izevbekhai and Shivraj Grewal for assistance with data collection, UCM Clinical Engineering for technical support, and Timothy Holper, MS, Julie Johnson, MPH, RN, and Thomas Sutton for assistance with data abstraction.

Disclosure

Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from Chest for invited speaking engagements. Dr. Churpek and Dr. Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and research support from EarlySense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. This study was supported by a grant from Philips Healthcare in Andover, MA. The sponsor had no role in data collection, interpretation of results, or drafting of the manuscript.

References

1. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):1170-1176. PubMed
2. Fieselmann JF, Hendryx MS, Helms CM, Wakefield DS. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med. 1993;8(7):354-360. PubMed
3. Semler MW, Stover DG, Copland AP, et al. Flash mob research: a single-day, multicenter, resident-directed study of respiratory rate. Chest. 2013;143(6):1740-1744. PubMed
4. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):1758-1765. PubMed
5. Edelson DP, Retzer E, Weidman EK, et al. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. J Hosp Med. 2011;6(8):475-479. PubMed
6. Patel AR, Zadravecz FJ, Young RS, Williams MV, Churpek MM, Edelson DP. The value of clinical judgment in the detection of clinical deterioration. JAMA Intern Med. 2015;175(3):456-458. PubMed

Article PDF
Issue
Journal of Hospital Medicine 13(7)
Publications
Topics
Page Number
486-489. Published online first February 2, 2018
Sections
Article PDF
Article PDF
Related Articles

Respiratory rate is the most accurate vital sign for predicting adverse outcomes in ward patients.1,2 Though other vital signs are typically collected by using machines, respiratory rate is collected manually by caregivers counting the breathing rate. However, studies have shown significant discrepancies between a patient’s respiratory rate documented in the medical record, which is often 18 or 20, and the value measured by counting the rate over a full minute.3 Thus, despite the high accuracy of respiratory rate, it is possible that these values do not represent true patient physiology. It is unknown whether a valid automated measurement of respiratory rate would be more predictive than a manually collected respiratory rate for identifying patients who develop deterioration. The aim of this study was to compare the distribution and predictive accuracy of manually and automatically recorded respiratory rates.

METHODS

In this prospective cohort study, adult patients admitted to one oncology ward at the University of Chicago from April 2015 to May 2016 were approached for consent (Institutional Review Board #14-0682). Enrolled patients were fit with a cableless, FDA-approved respiratory pod device (Philips IntelliVue clResp Pod; Philips Healthcare, Andover, MA) that automatically recorded respiratory rate and heart rate every 15 minutes while they remained on the ward. Pod data were paired with vital sign data documented in the electronic health record (EHR) by taking the automated value closest, but prior to, the manual value up to a maximum of 4 hours. Automated and manual respiratory rate were compared by using the area under the receiver operating characteristic curve (AUC) for whether an intensive care unit (ICU) transfer occurred within 24 hours of each paired observation without accounting for patient-level clustering.

RESULTS

A total of 1402 paired respiratory rate observations from 51 patient admissions were included, of which 5 patients (9.8%) experienced an ICU transfer. Paired heart rate values were highly correlated (r = 0.86), while paired respiratory rate values were less correlated (r = 0.38). The automated values had a median of 21 (interquartile range [IQR] of 17-25), while the manual values had a median of 18 (IQR of 16-21). Manual respiratory rates were significantly more accurate for predicting ICU transfer than automated respiratory rates (AUC 0.67 [95% CI, 0.62-0.73] vs 0.60 [95% CI, 0.55-0.65]; P = .011). As shown in the Figure, accuracy was similar between manual and automated respiratory rates until 18 breaths per minute, above which the manual respiratory rates were more predictive. At a threshold with similar specificity, manual respiratory rates >22 had a sensitivity of 45% and specificity of 84%, while automated respiratory rates >26 had a sensitivity of 22% and specificity of 81%. At a threshold with similar sensitivity, manual respiratory rates >20 had a sensitivity of 54% and specificity of 75%, while automated respiratory rates >22 had a sensitivity of 52% and specificity of 64%.

DISCUSSION

In this prospective cohort study, we found that manual respiratory rates were different than those collected from an automated system and, yet, were significantly more accurate for predicting ICU transfer. These results suggest that the predictive accuracy of respiratory rates documented in the EHR is due to more than just physiology. Our findings have important implications for the risk stratification of ward patients.

Though previous literature has suggested that respiratory rate is the most accurate predictor of deterioration, this may not be true.1 Respiratory rates manually recorded by clinical staff may contain information beyond pure physiology, such as a proxy of clinician concern, which may inflate the predictive value. Nursing staff may record standard respiratory rate values for patients that appear to be well (eg, 18) but count actual rates for those patients they suspect have a more severe disease, which is one possible explanation for our findings. In addition, automated assessments are likely to be more sensitive to intermittent fluctuations in respiratory rate associated with patient movement or emotion. This might explain the improved accuracy at higher rates for manually recorded vital signs.

Although limited by its small sample size, our results have important implications for patient monitoring and early warning scores designed to identify high-risk ward patients given that both simple scores and statistically derived models include respiratory rates as a predictor.4 As hospitals move to use newer technologies to automate vital sign monitoring and decrease nursing workload, our findings suggest that accuracy for identifying high-risk patients may be lost. Additional methods for capturing subjective assessments from clinical providers may be necessary and could be incorporated into risk scores.5 For example, the 7-point subjective Patient Acuity Rating has been shown to augment the Modified Early Warning Score for predicting ICU transfer, rapid response activation, or cardiac arrest within 24 hours.6

Manually recorded respiratory rate may include information beyond pure physiology, which inflates its predictive value. This has important implications for the use of automated monitoring technology in hospitals and the integration of these measurements into early warning scores.

 

 

Acknowledgments

The authors thank Pamela McCall, BSN, OCN for her assistance with study implementation, Kevin Ig-Izevbekhai and Shivraj Grewal for assistance with data collection, UCM Clinical Engineering for technical support, and Timothy Holper, MS, Julie Johnson, MPH, RN, and Thomas Sutton for assistance with data abstraction.

Disclosure

Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from Chest for invited speaking engagements. Dr. Churpek and Dr. Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and research support from EarlySense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. This study was supported by a grant from Philips Healthcare in Andover, MA. The sponsor had no role in data collection, interpretation of results, or drafting of the manuscript.

Respiratory rate is the most accurate vital sign for predicting adverse outcomes in ward patients.1,2 Though other vital signs are typically collected by using machines, respiratory rate is collected manually by caregivers counting the breathing rate. However, studies have shown significant discrepancies between a patient’s respiratory rate documented in the medical record, which is often 18 or 20, and the value measured by counting the rate over a full minute.3 Thus, despite the high accuracy of respiratory rate, it is possible that these values do not represent true patient physiology. It is unknown whether a valid automated measurement of respiratory rate would be more predictive than a manually collected respiratory rate for identifying patients who develop deterioration. The aim of this study was to compare the distribution and predictive accuracy of manually and automatically recorded respiratory rates.

METHODS

In this prospective cohort study, adult patients admitted to one oncology ward at the University of Chicago from April 2015 to May 2016 were approached for consent (Institutional Review Board #14-0682). Enrolled patients were fit with a cableless, FDA-approved respiratory pod device (Philips IntelliVue clResp Pod; Philips Healthcare, Andover, MA) that automatically recorded respiratory rate and heart rate every 15 minutes while they remained on the ward. Pod data were paired with vital sign data documented in the electronic health record (EHR) by taking the automated value closest, but prior to, the manual value up to a maximum of 4 hours. Automated and manual respiratory rate were compared by using the area under the receiver operating characteristic curve (AUC) for whether an intensive care unit (ICU) transfer occurred within 24 hours of each paired observation without accounting for patient-level clustering.

RESULTS

A total of 1402 paired respiratory rate observations from 51 patient admissions were included, of which 5 patients (9.8%) experienced an ICU transfer. Paired heart rate values were highly correlated (r = 0.86), while paired respiratory rate values were less correlated (r = 0.38). The automated values had a median of 21 (interquartile range [IQR] of 17-25), while the manual values had a median of 18 (IQR of 16-21). Manual respiratory rates were significantly more accurate for predicting ICU transfer than automated respiratory rates (AUC 0.67 [95% CI, 0.62-0.73] vs 0.60 [95% CI, 0.55-0.65]; P = .011). As shown in the Figure, accuracy was similar between manual and automated respiratory rates until 18 breaths per minute, above which the manual respiratory rates were more predictive. At a threshold with similar specificity, manual respiratory rates >22 had a sensitivity of 45% and specificity of 84%, while automated respiratory rates >26 had a sensitivity of 22% and specificity of 81%. At a threshold with similar sensitivity, manual respiratory rates >20 had a sensitivity of 54% and specificity of 75%, while automated respiratory rates >22 had a sensitivity of 52% and specificity of 64%.

DISCUSSION

In this prospective cohort study, we found that manual respiratory rates were different than those collected from an automated system and, yet, were significantly more accurate for predicting ICU transfer. These results suggest that the predictive accuracy of respiratory rates documented in the EHR is due to more than just physiology. Our findings have important implications for the risk stratification of ward patients.

Though previous literature has suggested that respiratory rate is the most accurate predictor of deterioration, this may not be true.1 Respiratory rates manually recorded by clinical staff may contain information beyond pure physiology, such as a proxy of clinician concern, which may inflate the predictive value. Nursing staff may record standard respiratory rate values for patients that appear to be well (eg, 18) but count actual rates for those patients they suspect have a more severe disease, which is one possible explanation for our findings. In addition, automated assessments are likely to be more sensitive to intermittent fluctuations in respiratory rate associated with patient movement or emotion. This might explain the improved accuracy at higher rates for manually recorded vital signs.

Although limited by its small sample size, our results have important implications for patient monitoring and early warning scores designed to identify high-risk ward patients given that both simple scores and statistically derived models include respiratory rates as a predictor.4 As hospitals move to use newer technologies to automate vital sign monitoring and decrease nursing workload, our findings suggest that accuracy for identifying high-risk patients may be lost. Additional methods for capturing subjective assessments from clinical providers may be necessary and could be incorporated into risk scores.5 For example, the 7-point subjective Patient Acuity Rating has been shown to augment the Modified Early Warning Score for predicting ICU transfer, rapid response activation, or cardiac arrest within 24 hours.6

Manually recorded respiratory rate may include information beyond pure physiology, which inflates its predictive value. This has important implications for the use of automated monitoring technology in hospitals and the integration of these measurements into early warning scores.

 

 

Acknowledgments

The authors thank Pamela McCall, BSN, OCN for her assistance with study implementation, Kevin Ig-Izevbekhai and Shivraj Grewal for assistance with data collection, UCM Clinical Engineering for technical support, and Timothy Holper, MS, Julie Johnson, MPH, RN, and Thomas Sutton for assistance with data abstraction.

Disclosure

Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from Chest for invited speaking engagements. Dr. Churpek and Dr. Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and research support from EarlySense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. This study was supported by a grant from Philips Healthcare in Andover, MA. The sponsor had no role in data collection, interpretation of results, or drafting of the manuscript.

References

1. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):1170-1176. PubMed
2. Fieselmann JF, Hendryx MS, Helms CM, Wakefield DS. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med. 1993;8(7):354-360. PubMed
3. Semler MW, Stover DG, Copland AP, et al. Flash mob research: a single-day, multicenter, resident-directed study of respiratory rate. Chest. 2013;143(6):1740-1744. PubMed
4. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):1758-1765. PubMed
5. Edelson DP, Retzer E, Weidman EK, et al. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. J Hosp Med. 2011;6(8):475-479. PubMed
6. Patel AR, Zadravecz FJ, Young RS, Williams MV, Churpek MM, Edelson DP. The value of clinical judgment in the detection of clinical deterioration. JAMA Intern Med. 2015;175(3):456-458. PubMed

References

1. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):1170-1176. PubMed
2. Fieselmann JF, Hendryx MS, Helms CM, Wakefield DS. Respiratory rate predicts cardiopulmonary arrest for internal medicine inpatients. J Gen Intern Med. 1993;8(7):354-360. PubMed
3. Semler MW, Stover DG, Copland AP, et al. Flash mob research: a single-day, multicenter, resident-directed study of respiratory rate. Chest. 2013;143(6):1740-1744. PubMed
4. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):1758-1765. PubMed
5. Edelson DP, Retzer E, Weidman EK, et al. Patient acuity rating: quantifying clinical judgment regarding inpatient stability. J Hosp Med. 2011;6(8):475-479. PubMed
6. Patel AR, Zadravecz FJ, Young RS, Williams MV, Churpek MM, Edelson DP. The value of clinical judgment in the detection of clinical deterioration. JAMA Intern Med. 2015;175(3):456-458. PubMed

Issue
Journal of Hospital Medicine 13(7)
Issue
Journal of Hospital Medicine 13(7)
Page Number
486-489. Published online first February 2, 2018
Page Number
486-489. Published online first February 2, 2018
Publications
Publications
Topics
Article Type
Sections
Article Source

© 2018 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Dana P. Edelson, MD, MS, 5841 South Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773-834-2191; Fax: 773-795-7398; E-mail: dperes@medicine.bsd.uchicago.edu
Content Gating
Gated (full article locked unless allowed per User)
Alternative CME
Disqus Comments
Default
Gate On Date
Wed, 08/15/2018 - 05:00
Un-Gate On Date
Wed, 07/11/2018 - 05:00
Use ProPublica
Gating Strategy
First Peek Free
Article PDF Media

Association between opioid and benzodiazepine use and clinical deterioration in ward patients

Article Type
Changed
Fri, 12/14/2018 - 08:17
Display Headline
Association between opioid and benzodiazepine use and clinical deterioration in ward patients

Chronic opioid and benzodiazepine use is common and increasing.1-5 Outpatient use of these medications has been associated with hospital readmission and death,6-12 with concurrent use associated with particularly increased risk.13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,15 many of which are new prescriptions rather than continuation of chronic therapy.16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.18 Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.22 Benzodiazepines also impair psychomotor function and recall,23 which could mediate the recognized risk for delirium and falls in the hospital.24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.31 If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents34 and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).34 Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.35

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities30 [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m2; normal, 18.5-24.9 kg/m2; overweight, 25.0-29.9 kg/m2; obese, 30-39.9 kg/m2; and severely obese, 40 mg/m2 or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity40 to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

Unadjusted frequency of composite outcome stratified by medication dose.
Figure

 

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Characteristics of Patient Admissions During Which Opioids and Benzodiazepines Were and Were Not Administered
Table 1

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Unadjusted Ward Outcome Rates for Patient Admissions With and Without Opioid or Benzodiazepine Administration
Table 2

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Adjusted Odds of Clinical Deterioration Outcomes Within Six Hours of Receiving an Opioid or Benzodiazepine
Table 3


Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.26,28 More recently, Herzig et al.15 found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.27 found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

Files
References

1. Substance Abuse and Mental Health Services Administration. Results from the 2013 National Survey on Drug Use and Health: Summary of National Findings. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2014.
2. Bachhuber MA, Hennessy S, Cunningham CO, Starrels JL. Increasing benzodiazepine prescriptions and overdose mortality in the United States, 1996–2013. Am J Public Health. 2016;106(4):686-688. PubMed
3. Parsells Kelly J, Cook SF, Kaufman DW, Anderson T, Rosenberg L, Mitchell AA. Prevalence and characteristics of opioid use in the US adult population. Pain. 2008;138(3):507-513. PubMed
4. Olfson M, King M, Schoenbaum M. Benzodiazepine use in the United States. JAMA Psychiatry. 2015;72(2):136-142. PubMed
5. Hwang CS, Kang EM, Kornegay CJ, Staffa JA, Jones CM, McAninch JK. Trends in the concomitant prescribing of opioids and benzodiazepines, 2002−2014. Am J Prev Med. 2016;51(2):151-160. PubMed
6. Bohnert AS, Valenstein M, Bair MJ, et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA. 2011;305(13):1315-1321. PubMed
7. Dart RC, Surratt HL, Cicero TJ, et al. Trends in opioid analgesic abuse and mortality in the United States. N Engl J Med. 2015;372(3):241-248. PubMed
8. Centers for Disease Control and Prevention (CDC). Vital signs: overdoses of prescription opioid pain relievers---United States, 1999--2008. MMWR Morb Mortal Wkly Rep. 2011;60(43):1487-1492. PubMed
9. Lan TY, Zeng YF, Tang GJ, et al. The use of hypnotics and mortality - a population-based retrospective cohort study. PLoS One. 2015;10(12):e0145271. PubMed
10. Mosher HJ, Jiang L, Vaughan Sarrazin MS, Cram P, Kaboli P, Vander Weg MW. Prevalence and characteristics of hospitalized adults on chronic opioid therapy: prior opioid use among veterans. J Hosp Med. 2014;9(2):82-87. PubMed
11. Palmaro A, Dupouy J, Lapeyre-Mestre M. Benzodiazepines and risk of death: results from two large cohort studies in France and UK. Eur Neuropsychopharmacol. 2015;25(10):1566-1577. PubMed
12. Parsaik AK, Mascarenhas SS, Khosh-Chashm D, et al. Mortality associated with anxiolytic and hypnotic drugs–a systematic review and meta-analysis. Aust N Z J Psychiatry. 2016;50(6):520-533. PubMed
13. Park TW, Saitz R, Ganoczy D, Ilgen MA, Bohnert AS. Benzodiazepine prescribing patterns and deaths from drug overdose among US veterans receiving opioid analgesics: case-cohort study. BMJ. 2015;350:h2698. PubMed
14. Jones CM, McAninch JK. Emergency department visits and overdose deaths from combined use of opioids and benzodiazepines. Am J Prev Med. 2015;49(4):493-501. PubMed
15. Herzig SJ, Rothberg MB, Cheung M, Ngo LH, Marcantonio ER. Opioid utilization and opioid-related adverse events in nonsurgical patients in US hospitals. J Hosp Med. 2014;9(2):73-81. PubMed
16. Jena AB, Goldman D, Karaca-Mandic P. Hospital prescribing of opioids to Medicare beneficiaries. JAMA Intern Med. 2016;176(7):990-997. PubMed
17. Calcaterra SL, Yamashita TE, Min SJ, Keniston A, Frank JW, Binswanger IA. Opioid prescribing at hospital discharge contributes to chronic opioid use. J Gen Intern Med. 2016;31(5):478-485. PubMed
18. Garrido MM, Prigerson HG, Penrod JD, Jones SC, Boockvar KS. Benzodiazepine and sedative-hypnotic use among older seriously ill veterans: choosing wisely? Clin Ther. 2014;36(11):1547-1554. PubMed
19. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PloS One. 2013;8(1):e54807. PubMed
20. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250-254. PubMed
21. Van Ryswyk E, Antic N. Opioids and sleep disordered breathing. Chest. 2016;150(4):934-944. PubMed
22. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diaz­epam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):65-69. PubMed
23. Pomara N, Lee SH, Bruno D, et al. Adverse performance effects of acute lorazepam administration in elderly long-term users: pharmacokinetic and clinical predictors. Prog Neuropsychopharmacol Biol Psychiatry. 2015;56:129-135. PubMed
24. Pandharipande P, Shintani A, Peterson J, et al. Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104(1):21-26. PubMed
25. O’Neil CA, Krauss MJ, Bettale J, et al. Medications and patient characteristics associated with falling in the hospital. J Patient Saf. 2015 (epub ahead of print). PubMed
26. Kessler ER, Shah M, K Gruschkus S, Raju A. Cost and quality implications of opioid-based postsurgical pain control using administrative claims data from a large health system: opioid-related adverse events and their impact on clinical and economic outcomes. Pharmacotherapy. 2013;33(4):383-391. PubMed
27. Overdyk FJ, Dowling O, Marino J, et al. Association of opioids and sedatives with increased risk of in-hospital cardiopulmonary arrest from an administrative database. PLoS One. 2016;11(2):e0150214. PubMed
28. Minkowitz HS, Gruschkus SK, Shah M, Raju A. Adverse drug events among patients receiving postsurgical opioids in a large health system: risk factors and outcomes. Am J Health Syst Pharm. 2014;71(18):1556-1565. PubMed
29. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. PubMed
30. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
31. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655. PubMed
32. Knaus WA, Wagner DP, Draper EA, Z et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619-1636. PubMed

33. van den Boogaard M, Pickkers P, Slooter AJC, et al. Development and validation
of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction
model for intensive care patients: observational multicentre study. BMJ.
2012;344:e420. PubMed
34. Clinical calculators. ClinCalc.com. http://www.clincalc.com. Accessed February
21, 2016.
35. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting
cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):
1170-1176. PubMed
36. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic
health record data to develop and validate a prediction model for adverse outcomes
in the wards. Crit Care Med. 2014;42(4):841-848. PubMed
37. Efron B. Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am
Stat Assoc. 1988;83(402):414-425.
38. Gibbons RD, Duan N, Meltzer D, et al; Institute of Medicine Committee. Waiting
for organ transplantation: results of an analysis by an Institute of Medicine Committee.
Biostatistics. 2003;4(2):207-222. PubMed
39. Singer JD, Willett JB. It’s about time: using discrete-time survival analysis to study
duration and the timing of events. J Educ Behav Stat. 1993;18(2):155-195.
40. World Health Organization. Cancer pain relief and palliative care. Report of a
WHO Expert Committee. World Health Organ Tech Rep Ser. 1990;804:1-75. PubMed
41. Bailey TC, Chen Y, Mao Y, et al. A trial of a real-time alert for clinical deterioration
in patients hospitalized on general medical wards. J Hosp Med. 2013;8(5):236-242. PubMed
42. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed
intensive care unit transfers in an integrated healthcare system. J Hosp Med.
2012;7(3):224-230. PubMed

 

 

 

 

 

Article PDF
Issue
Journal of Hospital Medicine 12(6)
Publications
Topics
Page Number
428-434
Sections
Files
Files
Article PDF
Article PDF

Chronic opioid and benzodiazepine use is common and increasing.1-5 Outpatient use of these medications has been associated with hospital readmission and death,6-12 with concurrent use associated with particularly increased risk.13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,15 many of which are new prescriptions rather than continuation of chronic therapy.16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.18 Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.22 Benzodiazepines also impair psychomotor function and recall,23 which could mediate the recognized risk for delirium and falls in the hospital.24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.31 If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents34 and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).34 Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.35

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities30 [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m2; normal, 18.5-24.9 kg/m2; overweight, 25.0-29.9 kg/m2; obese, 30-39.9 kg/m2; and severely obese, 40 mg/m2 or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity40 to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

Unadjusted frequency of composite outcome stratified by medication dose.
Figure

 

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Characteristics of Patient Admissions During Which Opioids and Benzodiazepines Were and Were Not Administered
Table 1

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Unadjusted Ward Outcome Rates for Patient Admissions With and Without Opioid or Benzodiazepine Administration
Table 2

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Adjusted Odds of Clinical Deterioration Outcomes Within Six Hours of Receiving an Opioid or Benzodiazepine
Table 3


Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.26,28 More recently, Herzig et al.15 found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.27 found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

Chronic opioid and benzodiazepine use is common and increasing.1-5 Outpatient use of these medications has been associated with hospital readmission and death,6-12 with concurrent use associated with particularly increased risk.13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,15 many of which are new prescriptions rather than continuation of chronic therapy.16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.18 Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.22 Benzodiazepines also impair psychomotor function and recall,23 which could mediate the recognized risk for delirium and falls in the hospital.24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.31 If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents34 and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).34 Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.35

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities30 [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m2; normal, 18.5-24.9 kg/m2; overweight, 25.0-29.9 kg/m2; obese, 30-39.9 kg/m2; and severely obese, 40 mg/m2 or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity40 to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

Unadjusted frequency of composite outcome stratified by medication dose.
Figure

 

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Characteristics of Patient Admissions During Which Opioids and Benzodiazepines Were and Were Not Administered
Table 1

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Unadjusted Ward Outcome Rates for Patient Admissions With and Without Opioid or Benzodiazepine Administration
Table 2

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Adjusted Odds of Clinical Deterioration Outcomes Within Six Hours of Receiving an Opioid or Benzodiazepine
Table 3


Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.26,28 More recently, Herzig et al.15 found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.27 found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

References

1. Substance Abuse and Mental Health Services Administration. Results from the 2013 National Survey on Drug Use and Health: Summary of National Findings. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2014.
2. Bachhuber MA, Hennessy S, Cunningham CO, Starrels JL. Increasing benzodiazepine prescriptions and overdose mortality in the United States, 1996–2013. Am J Public Health. 2016;106(4):686-688. PubMed
3. Parsells Kelly J, Cook SF, Kaufman DW, Anderson T, Rosenberg L, Mitchell AA. Prevalence and characteristics of opioid use in the US adult population. Pain. 2008;138(3):507-513. PubMed
4. Olfson M, King M, Schoenbaum M. Benzodiazepine use in the United States. JAMA Psychiatry. 2015;72(2):136-142. PubMed
5. Hwang CS, Kang EM, Kornegay CJ, Staffa JA, Jones CM, McAninch JK. Trends in the concomitant prescribing of opioids and benzodiazepines, 2002−2014. Am J Prev Med. 2016;51(2):151-160. PubMed
6. Bohnert AS, Valenstein M, Bair MJ, et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA. 2011;305(13):1315-1321. PubMed
7. Dart RC, Surratt HL, Cicero TJ, et al. Trends in opioid analgesic abuse and mortality in the United States. N Engl J Med. 2015;372(3):241-248. PubMed
8. Centers for Disease Control and Prevention (CDC). Vital signs: overdoses of prescription opioid pain relievers---United States, 1999--2008. MMWR Morb Mortal Wkly Rep. 2011;60(43):1487-1492. PubMed
9. Lan TY, Zeng YF, Tang GJ, et al. The use of hypnotics and mortality - a population-based retrospective cohort study. PLoS One. 2015;10(12):e0145271. PubMed
10. Mosher HJ, Jiang L, Vaughan Sarrazin MS, Cram P, Kaboli P, Vander Weg MW. Prevalence and characteristics of hospitalized adults on chronic opioid therapy: prior opioid use among veterans. J Hosp Med. 2014;9(2):82-87. PubMed
11. Palmaro A, Dupouy J, Lapeyre-Mestre M. Benzodiazepines and risk of death: results from two large cohort studies in France and UK. Eur Neuropsychopharmacol. 2015;25(10):1566-1577. PubMed
12. Parsaik AK, Mascarenhas SS, Khosh-Chashm D, et al. Mortality associated with anxiolytic and hypnotic drugs–a systematic review and meta-analysis. Aust N Z J Psychiatry. 2016;50(6):520-533. PubMed
13. Park TW, Saitz R, Ganoczy D, Ilgen MA, Bohnert AS. Benzodiazepine prescribing patterns and deaths from drug overdose among US veterans receiving opioid analgesics: case-cohort study. BMJ. 2015;350:h2698. PubMed
14. Jones CM, McAninch JK. Emergency department visits and overdose deaths from combined use of opioids and benzodiazepines. Am J Prev Med. 2015;49(4):493-501. PubMed
15. Herzig SJ, Rothberg MB, Cheung M, Ngo LH, Marcantonio ER. Opioid utilization and opioid-related adverse events in nonsurgical patients in US hospitals. J Hosp Med. 2014;9(2):73-81. PubMed
16. Jena AB, Goldman D, Karaca-Mandic P. Hospital prescribing of opioids to Medicare beneficiaries. JAMA Intern Med. 2016;176(7):990-997. PubMed
17. Calcaterra SL, Yamashita TE, Min SJ, Keniston A, Frank JW, Binswanger IA. Opioid prescribing at hospital discharge contributes to chronic opioid use. J Gen Intern Med. 2016;31(5):478-485. PubMed
18. Garrido MM, Prigerson HG, Penrod JD, Jones SC, Boockvar KS. Benzodiazepine and sedative-hypnotic use among older seriously ill veterans: choosing wisely? Clin Ther. 2014;36(11):1547-1554. PubMed
19. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PloS One. 2013;8(1):e54807. PubMed
20. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250-254. PubMed
21. Van Ryswyk E, Antic N. Opioids and sleep disordered breathing. Chest. 2016;150(4):934-944. PubMed
22. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diaz­epam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):65-69. PubMed
23. Pomara N, Lee SH, Bruno D, et al. Adverse performance effects of acute lorazepam administration in elderly long-term users: pharmacokinetic and clinical predictors. Prog Neuropsychopharmacol Biol Psychiatry. 2015;56:129-135. PubMed
24. Pandharipande P, Shintani A, Peterson J, et al. Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104(1):21-26. PubMed
25. O’Neil CA, Krauss MJ, Bettale J, et al. Medications and patient characteristics associated with falling in the hospital. J Patient Saf. 2015 (epub ahead of print). PubMed
26. Kessler ER, Shah M, K Gruschkus S, Raju A. Cost and quality implications of opioid-based postsurgical pain control using administrative claims data from a large health system: opioid-related adverse events and their impact on clinical and economic outcomes. Pharmacotherapy. 2013;33(4):383-391. PubMed
27. Overdyk FJ, Dowling O, Marino J, et al. Association of opioids and sedatives with increased risk of in-hospital cardiopulmonary arrest from an administrative database. PLoS One. 2016;11(2):e0150214. PubMed
28. Minkowitz HS, Gruschkus SK, Shah M, Raju A. Adverse drug events among patients receiving postsurgical opioids in a large health system: risk factors and outcomes. Am J Health Syst Pharm. 2014;71(18):1556-1565. PubMed
29. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. PubMed
30. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
31. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655. PubMed
32. Knaus WA, Wagner DP, Draper EA, Z et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619-1636. PubMed

33. van den Boogaard M, Pickkers P, Slooter AJC, et al. Development and validation
of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction
model for intensive care patients: observational multicentre study. BMJ.
2012;344:e420. PubMed
34. Clinical calculators. ClinCalc.com. http://www.clincalc.com. Accessed February
21, 2016.
35. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting
cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):
1170-1176. PubMed
36. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic
health record data to develop and validate a prediction model for adverse outcomes
in the wards. Crit Care Med. 2014;42(4):841-848. PubMed
37. Efron B. Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am
Stat Assoc. 1988;83(402):414-425.
38. Gibbons RD, Duan N, Meltzer D, et al; Institute of Medicine Committee. Waiting
for organ transplantation: results of an analysis by an Institute of Medicine Committee.
Biostatistics. 2003;4(2):207-222. PubMed
39. Singer JD, Willett JB. It’s about time: using discrete-time survival analysis to study
duration and the timing of events. J Educ Behav Stat. 1993;18(2):155-195.
40. World Health Organization. Cancer pain relief and palliative care. Report of a
WHO Expert Committee. World Health Organ Tech Rep Ser. 1990;804:1-75. PubMed
41. Bailey TC, Chen Y, Mao Y, et al. A trial of a real-time alert for clinical deterioration
in patients hospitalized on general medical wards. J Hosp Med. 2013;8(5):236-242. PubMed
42. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed
intensive care unit transfers in an integrated healthcare system. J Hosp Med.
2012;7(3):224-230. PubMed

 

 

 

 

 

References

1. Substance Abuse and Mental Health Services Administration. Results from the 2013 National Survey on Drug Use and Health: Summary of National Findings. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2014.
2. Bachhuber MA, Hennessy S, Cunningham CO, Starrels JL. Increasing benzodiazepine prescriptions and overdose mortality in the United States, 1996–2013. Am J Public Health. 2016;106(4):686-688. PubMed
3. Parsells Kelly J, Cook SF, Kaufman DW, Anderson T, Rosenberg L, Mitchell AA. Prevalence and characteristics of opioid use in the US adult population. Pain. 2008;138(3):507-513. PubMed
4. Olfson M, King M, Schoenbaum M. Benzodiazepine use in the United States. JAMA Psychiatry. 2015;72(2):136-142. PubMed
5. Hwang CS, Kang EM, Kornegay CJ, Staffa JA, Jones CM, McAninch JK. Trends in the concomitant prescribing of opioids and benzodiazepines, 2002−2014. Am J Prev Med. 2016;51(2):151-160. PubMed
6. Bohnert AS, Valenstein M, Bair MJ, et al. Association between opioid prescribing patterns and opioid overdose-related deaths. JAMA. 2011;305(13):1315-1321. PubMed
7. Dart RC, Surratt HL, Cicero TJ, et al. Trends in opioid analgesic abuse and mortality in the United States. N Engl J Med. 2015;372(3):241-248. PubMed
8. Centers for Disease Control and Prevention (CDC). Vital signs: overdoses of prescription opioid pain relievers---United States, 1999--2008. MMWR Morb Mortal Wkly Rep. 2011;60(43):1487-1492. PubMed
9. Lan TY, Zeng YF, Tang GJ, et al. The use of hypnotics and mortality - a population-based retrospective cohort study. PLoS One. 2015;10(12):e0145271. PubMed
10. Mosher HJ, Jiang L, Vaughan Sarrazin MS, Cram P, Kaboli P, Vander Weg MW. Prevalence and characteristics of hospitalized adults on chronic opioid therapy: prior opioid use among veterans. J Hosp Med. 2014;9(2):82-87. PubMed
11. Palmaro A, Dupouy J, Lapeyre-Mestre M. Benzodiazepines and risk of death: results from two large cohort studies in France and UK. Eur Neuropsychopharmacol. 2015;25(10):1566-1577. PubMed
12. Parsaik AK, Mascarenhas SS, Khosh-Chashm D, et al. Mortality associated with anxiolytic and hypnotic drugs–a systematic review and meta-analysis. Aust N Z J Psychiatry. 2016;50(6):520-533. PubMed
13. Park TW, Saitz R, Ganoczy D, Ilgen MA, Bohnert AS. Benzodiazepine prescribing patterns and deaths from drug overdose among US veterans receiving opioid analgesics: case-cohort study. BMJ. 2015;350:h2698. PubMed
14. Jones CM, McAninch JK. Emergency department visits and overdose deaths from combined use of opioids and benzodiazepines. Am J Prev Med. 2015;49(4):493-501. PubMed
15. Herzig SJ, Rothberg MB, Cheung M, Ngo LH, Marcantonio ER. Opioid utilization and opioid-related adverse events in nonsurgical patients in US hospitals. J Hosp Med. 2014;9(2):73-81. PubMed
16. Jena AB, Goldman D, Karaca-Mandic P. Hospital prescribing of opioids to Medicare beneficiaries. JAMA Intern Med. 2016;176(7):990-997. PubMed
17. Calcaterra SL, Yamashita TE, Min SJ, Keniston A, Frank JW, Binswanger IA. Opioid prescribing at hospital discharge contributes to chronic opioid use. J Gen Intern Med. 2016;31(5):478-485. PubMed
18. Garrido MM, Prigerson HG, Penrod JD, Jones SC, Boockvar KS. Benzodiazepine and sedative-hypnotic use among older seriously ill veterans: choosing wisely? Clin Ther. 2014;36(11):1547-1554. PubMed
19. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PloS One. 2013;8(1):e54807. PubMed
20. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250-254. PubMed
21. Van Ryswyk E, Antic N. Opioids and sleep disordered breathing. Chest. 2016;150(4):934-944. PubMed
22. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diaz­epam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):65-69. PubMed
23. Pomara N, Lee SH, Bruno D, et al. Adverse performance effects of acute lorazepam administration in elderly long-term users: pharmacokinetic and clinical predictors. Prog Neuropsychopharmacol Biol Psychiatry. 2015;56:129-135. PubMed
24. Pandharipande P, Shintani A, Peterson J, et al. Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients. Anesthesiology. 2006;104(1):21-26. PubMed
25. O’Neil CA, Krauss MJ, Bettale J, et al. Medications and patient characteristics associated with falling in the hospital. J Patient Saf. 2015 (epub ahead of print). PubMed
26. Kessler ER, Shah M, K Gruschkus S, Raju A. Cost and quality implications of opioid-based postsurgical pain control using administrative claims data from a large health system: opioid-related adverse events and their impact on clinical and economic outcomes. Pharmacotherapy. 2013;33(4):383-391. PubMed
27. Overdyk FJ, Dowling O, Marino J, et al. Association of opioids and sedatives with increased risk of in-hospital cardiopulmonary arrest from an administrative database. PLoS One. 2016;11(2):e0150214. PubMed
28. Minkowitz HS, Gruschkus SK, Shah M, Raju A. Adverse drug events among patients receiving postsurgical opioids in a large health system: risk factors and outcomes. Am J Health Syst Pharm. 2014;71(18):1556-1565. PubMed
29. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8-27. PubMed
30. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed
31. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655. PubMed
32. Knaus WA, Wagner DP, Draper EA, Z et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):1619-1636. PubMed

33. van den Boogaard M, Pickkers P, Slooter AJC, et al. Development and validation
of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction
model for intensive care patients: observational multicentre study. BMJ.
2012;344:e420. PubMed
34. Clinical calculators. ClinCalc.com. http://www.clincalc.com. Accessed February
21, 2016.
35. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting
cardiac arrest on the wards: a nested case-control study. Chest. 2012;141(5):
1170-1176. PubMed
36. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic
health record data to develop and validate a prediction model for adverse outcomes
in the wards. Crit Care Med. 2014;42(4):841-848. PubMed
37. Efron B. Logistic regression, survival analysis, and the Kaplan-Meier curve. J Am
Stat Assoc. 1988;83(402):414-425.
38. Gibbons RD, Duan N, Meltzer D, et al; Institute of Medicine Committee. Waiting
for organ transplantation: results of an analysis by an Institute of Medicine Committee.
Biostatistics. 2003;4(2):207-222. PubMed
39. Singer JD, Willett JB. It’s about time: using discrete-time survival analysis to study
duration and the timing of events. J Educ Behav Stat. 1993;18(2):155-195.
40. World Health Organization. Cancer pain relief and palliative care. Report of a
WHO Expert Committee. World Health Organ Tech Rep Ser. 1990;804:1-75. PubMed
41. Bailey TC, Chen Y, Mao Y, et al. A trial of a real-time alert for clinical deterioration
in patients hospitalized on general medical wards. J Hosp Med. 2013;8(5):236-242. PubMed
42. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed
intensive care unit transfers in an integrated healthcare system. J Hosp Med.
2012;7(3):224-230. PubMed

 

 

 

 

 

Issue
Journal of Hospital Medicine 12(6)
Issue
Journal of Hospital Medicine 12(6)
Page Number
428-434
Page Number
428-434
Publications
Publications
Topics
Article Type
Display Headline
Association between opioid and benzodiazepine use and clinical deterioration in ward patients
Display Headline
Association between opioid and benzodiazepine use and clinical deterioration in ward patients
Sections
Article Source

© 2017 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Matthew M. Churpek, MD, MPH, PhD, University of Chicago, Section of Pulmonary and Critical Care, 5841 S. Maryland Avenue, MC 6076, Chicago, IL 60637; Telephone: 773-702-1092; Fax: 773-702-6500; E-mail: matthew.churpek@uchospitals.edu



Content Gating
Open Access (article Unlocked/Open Access)
Alternative CME
Use ProPublica
Article PDF Media
Media Files

Detecting sepsis: Are two opinions better than one?

Article Type
Changed
Fri, 12/14/2018 - 08:27
Display Headline
Detecting sepsis: Are two opinions better than one?

Sepsis is a leading cause of hospital mortality in the United States, contributing to up to half of all deaths.1 If the infection is identified and treated early, however, its associated morbidity and mortality can be significantly reduced.2 The 2001 sepsis guidelines define sepsis as the suspicion of infection plus meeting 2 or more systemic inflammatory response syndrome (SIRS) criteria.3 Although the utility of SIRS criteria has been extensively debated, providers’ accuracy and agreement regarding suspicion of infection are not yet fully characterized. This is very important, as the source of infection is often not identified in patients with severe sepsis or septic shock.4

Although much attention recently has been given to ideal objective criteria for accurately identifying sepsis, less is known about what constitutes ideal subjective criteria and who can best make that assessment.5-7 We conducted a study to measure providers’ agreement regarding this subjective assessment and the impact of that agreement on patient outcomes.

METHODS

We performed a secondary analysis of prospectively collected data on consecutive adults hospitalized on a general medicine ward at an academic medical center between April 1, 2014 and March 31, 2015. This study was approved by the University of Chicago Institutional Review Board with a waiver of consent.

A sepsis screening tool was developed locally as part of the Surviving Sepsis Campaign Quality Improvement Learning Collaborative8 (Supplemental Figure). This tool was completed by bedside nurses for each patient during each shift. Bedside registered nurse (RN) suspicion of infection was deemed positive if the nurse answered yes to question 2: “Does the patient have evidence of an active infection?” We compared RN assessment with assessment by the ordering provider, a medical doctor or advanced practice professionals (MD/APP), using an existing order for antibiotics or a new order for either blood or urine cultures placed within 12 hours before nursing screen time to indicate MD/APP suspicion of infection.

All nursing screens were transcribed into an electronic database, excluding screens not performed, or missing RN suspicion of infection. For quality purposes, screening data were merged with electronic health record data to verify SIRS criteria at the time of the screens as well as the presence of culture and/or antibiotic orders preceding the screens. Outcome data were obtained from an administrative database and confirmed by chart review using the 2001 sepsis definitions.6 Data were de-identified and time-shifted before this analysis. SIRS-positive criteria were defined as meeting 2 or more of the following: temperature higher than 38°C or lower than 36°C; heart rate higher than 90 beats per minute; respiratory rate more than 20 breaths per minute; and white blood cell count more than 2,000/mm3 or less than 4,000/mm3.The primary clinical outcome was progression to severe sepsis or septic shock. Secondary outcomes included transfer to intensive care unit (ICU) and in-hospital mortality. Given that RN and MD/APP suspicion of infection can vary over time, only the initial screen for each patient was used in assessing progression to severe sepsis or septic shock and in-hospital mortality. All available screens were used to investigate the association between each provider’s suspicion of infection over time and ICU transfer.

Demographic characteristics were compared using the χ2 test and analysis of variance, as appropriate. Provider agreement was evaluated with a weighted κ statistic. Fisher exact tests were used to compare proportions of mortality and severe sepsis/septic shock, and the McNemar test was used to compare proportions of ICU transfers. The association of outcomes based on provider agreement was evaluated with a nonparametric test for trend.

 

 

RESULTS

During the study period, 1386 distinct patients had 13,223 screening opportunities, with a 95.4% compliance rate. A total of 1127 screens were excluded for missing nursing documentation of suspicion of infection, leaving 1192 first screens and 11,489 total screens for analysis. Of the completed screens, 3744 (32.6%) met SIRS criteria; suspicion of infection was noted by both RN and MD/APP in 5.8% of cases, by RN only in 22.2%, by MD/APP only in 7.2%, and by neither provider in 64.7% (Figure 1). Overall agreement rate was 80.7% for suspicion of infection (κ = 0.11, P < 0.001). Demographics by subgroup are shown in the Supplemental Table. Progression to severe sepsis or shock was highest when both providers suspected infection in a SIRS-positive patient (17.7%), was substantially reduced with single-provider suspicion (6.0%), and was lowest when neither provider suspected infection (1.5%) (P < 0.001). A similar trend was found for in-hospital mortality (both providers, 6.3%; single provider, 2.7%; neither provider, 2.5%; P = 0.01). Compared with MD/APP-only suspicion, SIRS-positive patients in whom only RNs suspected infection had similar frequency of progression to severe sepsis or septic shock (6.5% vs 5.6%; P = 0.52) and higher mortality (5.0% vs 1.1%; P = 0.32), though these findings were not statistically significant.

Provider agreement on suspicion of infection in patients meeting 2 of 4 SIRS criteria.
Figure 1

For the 121 patients (10.2%) transferred to ICU, RNs were more likely than MD/APPs to suspect infection at all time points (Figure 2). The difference was small (P = 0.29) 48 hours before transfer (RN, 12.5%; MD/APP, 5.6%) but became more pronounced (P = 0.06) by 3 hours before transfer (RN, 46.3%; MD/APP, 33.1%). Nursing assessments were not available after transfer, but 3 hours after transfer the proportion of patients who met MD/APP suspicion-of-infection criteria (44.6%) was similar (P = 0.90) to that of the RNs 3 hours before transfer (46.3%).

Cumulative suspicion of infection by provider over time in patients transferred to ICU.
Figure 2

DISCUSSION

Our findings reveal that bedside nurses and ordering providers routinely have discordant assessments regarding presence of infection. Specifically, when RNs are asked to screen patients on the wards, they are suspicious of infection more often than MD/APPs are, and they suspect infection earlier in ICU transfer patients. These findings have significant implications for patient care, compliance with the new national SEP-1 Centers for Medicare & Medicaid Services quality measure, and identification of appropriate patients for enrollment in sepsis-related clinical trials.

To our knowledge, this is the first study to explore agreement between bedside RN and MD/APP suspicion of infection in sepsis screening and its association with patient outcomes. Studies on nurse and physician concordance in other domains have had mixed findings.9-11 The high discordance rate found in our study points to the highly subjective nature of suspicion of infection.

Our finding that RNs suspect infection earlier in patients transferred to ICU suggests nursing suspicion has value above and beyond current practice. A possible explanation for the higher rate of RN suspicion, and earlier RN suspicion, is that bedside nurses spend substantially more time with their patients and are more attuned to subtle changes that often occur before any objective signs of deterioration. This phenomenon is well documented and accounts for why rapid response calling criteria often include “nurse worry or concern.”12,13 Thus, nurse intuition may be an important signal for early identification of patients at high risk for sepsis.

That about one third of all screens met SIRS criteria and that almost two thirds of those screens were not thought by RN or MD/APP to be caused by infection add to the literature demonstrating the limited value of SIRS as a screening tool for sepsis.14 To address this issue, the 2016 sepsis definitions propose using the quick Sepsis-Related Organ Failure Assessment (qSOFA) to identify patients at high risk for clinical deterioration; however, the Surviving Sepsis Campaign continues to encourage sepsis screening using the SIRS criteria.15

Limitations of this study include its lack of generalizability, as it was conducted with general medical patients at a single center. Second, we did not specifically ask the MD/APPs whether they suspected infection; instead, we relied on their ordering practices. Third, RN and MD/APP assessments were not independent, as RNs had access to MD/APP orders before making their own assessments, which could bias our results.

Discordance in provider suspicion of infection is common, with RNs documenting suspicion more often than MD/APPs, and earlier in patients transferred to ICU. Suspicion by either provider alone is associated with higher risk for sepsis progression and in-hospital mortality than is the case when neither provider suspects infection. Thus, a collaborative method that includes both RNs and MD/APPs may improve the accuracy and timing of sepsis detection on the wards.

 

 

Acknowledgments

The authors thank the members of the Surviving Sepsis Campaign (SSC) Quality Improvement Learning Collaborative at the University of Chicago for their help in data collection and review, especially Meredith Borak, Rita Lanier, Mary Ann Francisco, and Bill Marsack. The authors also thank Thomas Best and Mary-Kate Springman for their assistance in data entry and Nicole Twu for administrative support. Data from this study were provided by the Clinical Research Data Warehouse (CRDW) maintained by the Center for Research Informatics (CRI) at the University of Chicago. CRI is funded by the Biological Sciences Division of the Institute for Translational Medicine/Clinical and Translational Science Award (CTSA) (National Institutes of Health UL1 TR000430) at the University of Chicago.

Disclosures

Dr. Bhattacharjee is supported by postdoctoral training grant 4T32HS000078 from the Agency for Healthcare Research and Quality. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by career development award K08 HL121080 from the National Heart, Lung, and Blood Institute. Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), American Heart Association (Dallas, Texas), and Laerdal Medical (Stavanger, Norway) and has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. The other authors report no conflicts of interest.

Files
References

1. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
2. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368-1377. PubMed
3. Levy MM, Fink MP, Marshall JC, et al; SCCM/ESICM/ACCP/ATS/SIS. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250-1256. PubMed
4. Vincent JL, Sakr Y, Sprung CL, et al; Sepsis Occurrence in Acutely Ill Patients Investigators. Sepsis in European intensive care units: results of the SOAP study. Crit Care Med. 2006;34(2):344-353. PubMed
5. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372(17):1629-1638. PubMed
6. Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change. Lancet. 2013;381(9868):774-775. PubMed
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
8. Surviving Sepsis Campaign (SSC) Sepsis on the Floors Quality Improvement Learning Collaborative. Frequently asked questions (FAQs). Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/About-Collaboratives.pdf. Published October 8, 2013.
9. Fiesseler F, Szucs P, Kec R, Richman PB. Can nurses appropriately interpret the Ottawa ankle rule? Am J Emerg Med. 2004;22(3):145-148. PubMed
10. Blomberg H, Lundström E, Toss H, Gedeborg R, Johansson J. Agreement between ambulance nurses and physicians in assessing stroke patients. Acta Neurol Scand. 2014;129(1):49­55. PubMed
11. Neville TH, Wiley JF, Yamamoto MC, et al. Concordance of nurses and physicians on whether critical care patients are receiving futile treatment. Am J Crit Care. 2015;24(5):403­410. PubMed
12. Odell M, Victor C, Oliver D. Nurses’ role in detecting deterioration in ward patients: systematic literature review. J Adv Nurs. 2009;65(10):1992-2006. PubMed
13. Howell MD, Ngo L, Folcarelli P, et al. Sustained effectiveness of a primary-team-based rapid response system. Crit Care Med. 2012;40(9):2562-2568. PubMed
14. Churpek MM, Zadravecz FJ, Winslow C, Howell MD, Edelson DP. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med. 2015;192(8):958-964. PubMed
15. Antonelli M, DeBacker D, Dorman T, Kleinpell R, Levy M, Rhodes A; Surviving Sepsis Campaign Executive Committee. Surviving Sepsis Campaign responds to Sepsis-3. Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/SSC-Statements-Sepsis-Definitions-3-2016.pdf. Published March 1, 2016. Accessed May 11, 2016.

Article PDF
Issue
Journal of Hospital Medicine 12(4)
Publications
Topics
Page Number
256-258
Sections
Files
Files
Article PDF
Article PDF

Sepsis is a leading cause of hospital mortality in the United States, contributing to up to half of all deaths.1 If the infection is identified and treated early, however, its associated morbidity and mortality can be significantly reduced.2 The 2001 sepsis guidelines define sepsis as the suspicion of infection plus meeting 2 or more systemic inflammatory response syndrome (SIRS) criteria.3 Although the utility of SIRS criteria has been extensively debated, providers’ accuracy and agreement regarding suspicion of infection are not yet fully characterized. This is very important, as the source of infection is often not identified in patients with severe sepsis or septic shock.4

Although much attention recently has been given to ideal objective criteria for accurately identifying sepsis, less is known about what constitutes ideal subjective criteria and who can best make that assessment.5-7 We conducted a study to measure providers’ agreement regarding this subjective assessment and the impact of that agreement on patient outcomes.

METHODS

We performed a secondary analysis of prospectively collected data on consecutive adults hospitalized on a general medicine ward at an academic medical center between April 1, 2014 and March 31, 2015. This study was approved by the University of Chicago Institutional Review Board with a waiver of consent.

A sepsis screening tool was developed locally as part of the Surviving Sepsis Campaign Quality Improvement Learning Collaborative8 (Supplemental Figure). This tool was completed by bedside nurses for each patient during each shift. Bedside registered nurse (RN) suspicion of infection was deemed positive if the nurse answered yes to question 2: “Does the patient have evidence of an active infection?” We compared RN assessment with assessment by the ordering provider, a medical doctor or advanced practice professionals (MD/APP), using an existing order for antibiotics or a new order for either blood or urine cultures placed within 12 hours before nursing screen time to indicate MD/APP suspicion of infection.

All nursing screens were transcribed into an electronic database, excluding screens not performed, or missing RN suspicion of infection. For quality purposes, screening data were merged with electronic health record data to verify SIRS criteria at the time of the screens as well as the presence of culture and/or antibiotic orders preceding the screens. Outcome data were obtained from an administrative database and confirmed by chart review using the 2001 sepsis definitions.6 Data were de-identified and time-shifted before this analysis. SIRS-positive criteria were defined as meeting 2 or more of the following: temperature higher than 38°C or lower than 36°C; heart rate higher than 90 beats per minute; respiratory rate more than 20 breaths per minute; and white blood cell count more than 2,000/mm3 or less than 4,000/mm3.The primary clinical outcome was progression to severe sepsis or septic shock. Secondary outcomes included transfer to intensive care unit (ICU) and in-hospital mortality. Given that RN and MD/APP suspicion of infection can vary over time, only the initial screen for each patient was used in assessing progression to severe sepsis or septic shock and in-hospital mortality. All available screens were used to investigate the association between each provider’s suspicion of infection over time and ICU transfer.

Demographic characteristics were compared using the χ2 test and analysis of variance, as appropriate. Provider agreement was evaluated with a weighted κ statistic. Fisher exact tests were used to compare proportions of mortality and severe sepsis/septic shock, and the McNemar test was used to compare proportions of ICU transfers. The association of outcomes based on provider agreement was evaluated with a nonparametric test for trend.

 

 

RESULTS

During the study period, 1386 distinct patients had 13,223 screening opportunities, with a 95.4% compliance rate. A total of 1127 screens were excluded for missing nursing documentation of suspicion of infection, leaving 1192 first screens and 11,489 total screens for analysis. Of the completed screens, 3744 (32.6%) met SIRS criteria; suspicion of infection was noted by both RN and MD/APP in 5.8% of cases, by RN only in 22.2%, by MD/APP only in 7.2%, and by neither provider in 64.7% (Figure 1). Overall agreement rate was 80.7% for suspicion of infection (κ = 0.11, P < 0.001). Demographics by subgroup are shown in the Supplemental Table. Progression to severe sepsis or shock was highest when both providers suspected infection in a SIRS-positive patient (17.7%), was substantially reduced with single-provider suspicion (6.0%), and was lowest when neither provider suspected infection (1.5%) (P < 0.001). A similar trend was found for in-hospital mortality (both providers, 6.3%; single provider, 2.7%; neither provider, 2.5%; P = 0.01). Compared with MD/APP-only suspicion, SIRS-positive patients in whom only RNs suspected infection had similar frequency of progression to severe sepsis or septic shock (6.5% vs 5.6%; P = 0.52) and higher mortality (5.0% vs 1.1%; P = 0.32), though these findings were not statistically significant.

Provider agreement on suspicion of infection in patients meeting 2 of 4 SIRS criteria.
Figure 1

For the 121 patients (10.2%) transferred to ICU, RNs were more likely than MD/APPs to suspect infection at all time points (Figure 2). The difference was small (P = 0.29) 48 hours before transfer (RN, 12.5%; MD/APP, 5.6%) but became more pronounced (P = 0.06) by 3 hours before transfer (RN, 46.3%; MD/APP, 33.1%). Nursing assessments were not available after transfer, but 3 hours after transfer the proportion of patients who met MD/APP suspicion-of-infection criteria (44.6%) was similar (P = 0.90) to that of the RNs 3 hours before transfer (46.3%).

Cumulative suspicion of infection by provider over time in patients transferred to ICU.
Figure 2

DISCUSSION

Our findings reveal that bedside nurses and ordering providers routinely have discordant assessments regarding presence of infection. Specifically, when RNs are asked to screen patients on the wards, they are suspicious of infection more often than MD/APPs are, and they suspect infection earlier in ICU transfer patients. These findings have significant implications for patient care, compliance with the new national SEP-1 Centers for Medicare & Medicaid Services quality measure, and identification of appropriate patients for enrollment in sepsis-related clinical trials.

To our knowledge, this is the first study to explore agreement between bedside RN and MD/APP suspicion of infection in sepsis screening and its association with patient outcomes. Studies on nurse and physician concordance in other domains have had mixed findings.9-11 The high discordance rate found in our study points to the highly subjective nature of suspicion of infection.

Our finding that RNs suspect infection earlier in patients transferred to ICU suggests nursing suspicion has value above and beyond current practice. A possible explanation for the higher rate of RN suspicion, and earlier RN suspicion, is that bedside nurses spend substantially more time with their patients and are more attuned to subtle changes that often occur before any objective signs of deterioration. This phenomenon is well documented and accounts for why rapid response calling criteria often include “nurse worry or concern.”12,13 Thus, nurse intuition may be an important signal for early identification of patients at high risk for sepsis.

That about one third of all screens met SIRS criteria and that almost two thirds of those screens were not thought by RN or MD/APP to be caused by infection add to the literature demonstrating the limited value of SIRS as a screening tool for sepsis.14 To address this issue, the 2016 sepsis definitions propose using the quick Sepsis-Related Organ Failure Assessment (qSOFA) to identify patients at high risk for clinical deterioration; however, the Surviving Sepsis Campaign continues to encourage sepsis screening using the SIRS criteria.15

Limitations of this study include its lack of generalizability, as it was conducted with general medical patients at a single center. Second, we did not specifically ask the MD/APPs whether they suspected infection; instead, we relied on their ordering practices. Third, RN and MD/APP assessments were not independent, as RNs had access to MD/APP orders before making their own assessments, which could bias our results.

Discordance in provider suspicion of infection is common, with RNs documenting suspicion more often than MD/APPs, and earlier in patients transferred to ICU. Suspicion by either provider alone is associated with higher risk for sepsis progression and in-hospital mortality than is the case when neither provider suspects infection. Thus, a collaborative method that includes both RNs and MD/APPs may improve the accuracy and timing of sepsis detection on the wards.

 

 

Acknowledgments

The authors thank the members of the Surviving Sepsis Campaign (SSC) Quality Improvement Learning Collaborative at the University of Chicago for their help in data collection and review, especially Meredith Borak, Rita Lanier, Mary Ann Francisco, and Bill Marsack. The authors also thank Thomas Best and Mary-Kate Springman for their assistance in data entry and Nicole Twu for administrative support. Data from this study were provided by the Clinical Research Data Warehouse (CRDW) maintained by the Center for Research Informatics (CRI) at the University of Chicago. CRI is funded by the Biological Sciences Division of the Institute for Translational Medicine/Clinical and Translational Science Award (CTSA) (National Institutes of Health UL1 TR000430) at the University of Chicago.

Disclosures

Dr. Bhattacharjee is supported by postdoctoral training grant 4T32HS000078 from the Agency for Healthcare Research and Quality. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by career development award K08 HL121080 from the National Heart, Lung, and Blood Institute. Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), American Heart Association (Dallas, Texas), and Laerdal Medical (Stavanger, Norway) and has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. The other authors report no conflicts of interest.

Sepsis is a leading cause of hospital mortality in the United States, contributing to up to half of all deaths.1 If the infection is identified and treated early, however, its associated morbidity and mortality can be significantly reduced.2 The 2001 sepsis guidelines define sepsis as the suspicion of infection plus meeting 2 or more systemic inflammatory response syndrome (SIRS) criteria.3 Although the utility of SIRS criteria has been extensively debated, providers’ accuracy and agreement regarding suspicion of infection are not yet fully characterized. This is very important, as the source of infection is often not identified in patients with severe sepsis or septic shock.4

Although much attention recently has been given to ideal objective criteria for accurately identifying sepsis, less is known about what constitutes ideal subjective criteria and who can best make that assessment.5-7 We conducted a study to measure providers’ agreement regarding this subjective assessment and the impact of that agreement on patient outcomes.

METHODS

We performed a secondary analysis of prospectively collected data on consecutive adults hospitalized on a general medicine ward at an academic medical center between April 1, 2014 and March 31, 2015. This study was approved by the University of Chicago Institutional Review Board with a waiver of consent.

A sepsis screening tool was developed locally as part of the Surviving Sepsis Campaign Quality Improvement Learning Collaborative8 (Supplemental Figure). This tool was completed by bedside nurses for each patient during each shift. Bedside registered nurse (RN) suspicion of infection was deemed positive if the nurse answered yes to question 2: “Does the patient have evidence of an active infection?” We compared RN assessment with assessment by the ordering provider, a medical doctor or advanced practice professionals (MD/APP), using an existing order for antibiotics or a new order for either blood or urine cultures placed within 12 hours before nursing screen time to indicate MD/APP suspicion of infection.

All nursing screens were transcribed into an electronic database, excluding screens not performed, or missing RN suspicion of infection. For quality purposes, screening data were merged with electronic health record data to verify SIRS criteria at the time of the screens as well as the presence of culture and/or antibiotic orders preceding the screens. Outcome data were obtained from an administrative database and confirmed by chart review using the 2001 sepsis definitions.6 Data were de-identified and time-shifted before this analysis. SIRS-positive criteria were defined as meeting 2 or more of the following: temperature higher than 38°C or lower than 36°C; heart rate higher than 90 beats per minute; respiratory rate more than 20 breaths per minute; and white blood cell count more than 2,000/mm3 or less than 4,000/mm3.The primary clinical outcome was progression to severe sepsis or septic shock. Secondary outcomes included transfer to intensive care unit (ICU) and in-hospital mortality. Given that RN and MD/APP suspicion of infection can vary over time, only the initial screen for each patient was used in assessing progression to severe sepsis or septic shock and in-hospital mortality. All available screens were used to investigate the association between each provider’s suspicion of infection over time and ICU transfer.

Demographic characteristics were compared using the χ2 test and analysis of variance, as appropriate. Provider agreement was evaluated with a weighted κ statistic. Fisher exact tests were used to compare proportions of mortality and severe sepsis/septic shock, and the McNemar test was used to compare proportions of ICU transfers. The association of outcomes based on provider agreement was evaluated with a nonparametric test for trend.

 

 

RESULTS

During the study period, 1386 distinct patients had 13,223 screening opportunities, with a 95.4% compliance rate. A total of 1127 screens were excluded for missing nursing documentation of suspicion of infection, leaving 1192 first screens and 11,489 total screens for analysis. Of the completed screens, 3744 (32.6%) met SIRS criteria; suspicion of infection was noted by both RN and MD/APP in 5.8% of cases, by RN only in 22.2%, by MD/APP only in 7.2%, and by neither provider in 64.7% (Figure 1). Overall agreement rate was 80.7% for suspicion of infection (κ = 0.11, P < 0.001). Demographics by subgroup are shown in the Supplemental Table. Progression to severe sepsis or shock was highest when both providers suspected infection in a SIRS-positive patient (17.7%), was substantially reduced with single-provider suspicion (6.0%), and was lowest when neither provider suspected infection (1.5%) (P < 0.001). A similar trend was found for in-hospital mortality (both providers, 6.3%; single provider, 2.7%; neither provider, 2.5%; P = 0.01). Compared with MD/APP-only suspicion, SIRS-positive patients in whom only RNs suspected infection had similar frequency of progression to severe sepsis or septic shock (6.5% vs 5.6%; P = 0.52) and higher mortality (5.0% vs 1.1%; P = 0.32), though these findings were not statistically significant.

Provider agreement on suspicion of infection in patients meeting 2 of 4 SIRS criteria.
Figure 1

For the 121 patients (10.2%) transferred to ICU, RNs were more likely than MD/APPs to suspect infection at all time points (Figure 2). The difference was small (P = 0.29) 48 hours before transfer (RN, 12.5%; MD/APP, 5.6%) but became more pronounced (P = 0.06) by 3 hours before transfer (RN, 46.3%; MD/APP, 33.1%). Nursing assessments were not available after transfer, but 3 hours after transfer the proportion of patients who met MD/APP suspicion-of-infection criteria (44.6%) was similar (P = 0.90) to that of the RNs 3 hours before transfer (46.3%).

Cumulative suspicion of infection by provider over time in patients transferred to ICU.
Figure 2

DISCUSSION

Our findings reveal that bedside nurses and ordering providers routinely have discordant assessments regarding presence of infection. Specifically, when RNs are asked to screen patients on the wards, they are suspicious of infection more often than MD/APPs are, and they suspect infection earlier in ICU transfer patients. These findings have significant implications for patient care, compliance with the new national SEP-1 Centers for Medicare & Medicaid Services quality measure, and identification of appropriate patients for enrollment in sepsis-related clinical trials.

To our knowledge, this is the first study to explore agreement between bedside RN and MD/APP suspicion of infection in sepsis screening and its association with patient outcomes. Studies on nurse and physician concordance in other domains have had mixed findings.9-11 The high discordance rate found in our study points to the highly subjective nature of suspicion of infection.

Our finding that RNs suspect infection earlier in patients transferred to ICU suggests nursing suspicion has value above and beyond current practice. A possible explanation for the higher rate of RN suspicion, and earlier RN suspicion, is that bedside nurses spend substantially more time with their patients and are more attuned to subtle changes that often occur before any objective signs of deterioration. This phenomenon is well documented and accounts for why rapid response calling criteria often include “nurse worry or concern.”12,13 Thus, nurse intuition may be an important signal for early identification of patients at high risk for sepsis.

That about one third of all screens met SIRS criteria and that almost two thirds of those screens were not thought by RN or MD/APP to be caused by infection add to the literature demonstrating the limited value of SIRS as a screening tool for sepsis.14 To address this issue, the 2016 sepsis definitions propose using the quick Sepsis-Related Organ Failure Assessment (qSOFA) to identify patients at high risk for clinical deterioration; however, the Surviving Sepsis Campaign continues to encourage sepsis screening using the SIRS criteria.15

Limitations of this study include its lack of generalizability, as it was conducted with general medical patients at a single center. Second, we did not specifically ask the MD/APPs whether they suspected infection; instead, we relied on their ordering practices. Third, RN and MD/APP assessments were not independent, as RNs had access to MD/APP orders before making their own assessments, which could bias our results.

Discordance in provider suspicion of infection is common, with RNs documenting suspicion more often than MD/APPs, and earlier in patients transferred to ICU. Suspicion by either provider alone is associated with higher risk for sepsis progression and in-hospital mortality than is the case when neither provider suspects infection. Thus, a collaborative method that includes both RNs and MD/APPs may improve the accuracy and timing of sepsis detection on the wards.

 

 

Acknowledgments

The authors thank the members of the Surviving Sepsis Campaign (SSC) Quality Improvement Learning Collaborative at the University of Chicago for their help in data collection and review, especially Meredith Borak, Rita Lanier, Mary Ann Francisco, and Bill Marsack. The authors also thank Thomas Best and Mary-Kate Springman for their assistance in data entry and Nicole Twu for administrative support. Data from this study were provided by the Clinical Research Data Warehouse (CRDW) maintained by the Center for Research Informatics (CRI) at the University of Chicago. CRI is funded by the Biological Sciences Division of the Institute for Translational Medicine/Clinical and Translational Science Award (CTSA) (National Institutes of Health UL1 TR000430) at the University of Chicago.

Disclosures

Dr. Bhattacharjee is supported by postdoctoral training grant 4T32HS000078 from the Agency for Healthcare Research and Quality. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by career development award K08 HL121080 from the National Heart, Lung, and Blood Institute. Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), American Heart Association (Dallas, Texas), and Laerdal Medical (Stavanger, Norway) and has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. The other authors report no conflicts of interest.

References

1. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
2. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368-1377. PubMed
3. Levy MM, Fink MP, Marshall JC, et al; SCCM/ESICM/ACCP/ATS/SIS. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250-1256. PubMed
4. Vincent JL, Sakr Y, Sprung CL, et al; Sepsis Occurrence in Acutely Ill Patients Investigators. Sepsis in European intensive care units: results of the SOAP study. Crit Care Med. 2006;34(2):344-353. PubMed
5. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372(17):1629-1638. PubMed
6. Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change. Lancet. 2013;381(9868):774-775. PubMed
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
8. Surviving Sepsis Campaign (SSC) Sepsis on the Floors Quality Improvement Learning Collaborative. Frequently asked questions (FAQs). Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/About-Collaboratives.pdf. Published October 8, 2013.
9. Fiesseler F, Szucs P, Kec R, Richman PB. Can nurses appropriately interpret the Ottawa ankle rule? Am J Emerg Med. 2004;22(3):145-148. PubMed
10. Blomberg H, Lundström E, Toss H, Gedeborg R, Johansson J. Agreement between ambulance nurses and physicians in assessing stroke patients. Acta Neurol Scand. 2014;129(1):49­55. PubMed
11. Neville TH, Wiley JF, Yamamoto MC, et al. Concordance of nurses and physicians on whether critical care patients are receiving futile treatment. Am J Crit Care. 2015;24(5):403­410. PubMed
12. Odell M, Victor C, Oliver D. Nurses’ role in detecting deterioration in ward patients: systematic literature review. J Adv Nurs. 2009;65(10):1992-2006. PubMed
13. Howell MD, Ngo L, Folcarelli P, et al. Sustained effectiveness of a primary-team-based rapid response system. Crit Care Med. 2012;40(9):2562-2568. PubMed
14. Churpek MM, Zadravecz FJ, Winslow C, Howell MD, Edelson DP. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med. 2015;192(8):958-964. PubMed
15. Antonelli M, DeBacker D, Dorman T, Kleinpell R, Levy M, Rhodes A; Surviving Sepsis Campaign Executive Committee. Surviving Sepsis Campaign responds to Sepsis-3. Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/SSC-Statements-Sepsis-Definitions-3-2016.pdf. Published March 1, 2016. Accessed May 11, 2016.

References

1. Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90-92. PubMed
2. Rivers E, Nguyen B, Havstad S, et al; Early Goal-Directed Therapy Collaborative Group. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368-1377. PubMed
3. Levy MM, Fink MP, Marshall JC, et al; SCCM/ESICM/ACCP/ATS/SIS. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med. 2003;31(4):1250-1256. PubMed
4. Vincent JL, Sakr Y, Sprung CL, et al; Sepsis Occurrence in Acutely Ill Patients Investigators. Sepsis in European intensive care units: results of the SOAP study. Crit Care Med. 2006;34(2):344-353. PubMed
5. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372(17):1629-1638. PubMed
6. Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change. Lancet. 2013;381(9868):774-775. PubMed
7. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-810. PubMed
8. Surviving Sepsis Campaign (SSC) Sepsis on the Floors Quality Improvement Learning Collaborative. Frequently asked questions (FAQs). Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/About-Collaboratives.pdf. Published October 8, 2013.
9. Fiesseler F, Szucs P, Kec R, Richman PB. Can nurses appropriately interpret the Ottawa ankle rule? Am J Emerg Med. 2004;22(3):145-148. PubMed
10. Blomberg H, Lundström E, Toss H, Gedeborg R, Johansson J. Agreement between ambulance nurses and physicians in assessing stroke patients. Acta Neurol Scand. 2014;129(1):49­55. PubMed
11. Neville TH, Wiley JF, Yamamoto MC, et al. Concordance of nurses and physicians on whether critical care patients are receiving futile treatment. Am J Crit Care. 2015;24(5):403­410. PubMed
12. Odell M, Victor C, Oliver D. Nurses’ role in detecting deterioration in ward patients: systematic literature review. J Adv Nurs. 2009;65(10):1992-2006. PubMed
13. Howell MD, Ngo L, Folcarelli P, et al. Sustained effectiveness of a primary-team-based rapid response system. Crit Care Med. 2012;40(9):2562-2568. PubMed
14. Churpek MM, Zadravecz FJ, Winslow C, Howell MD, Edelson DP. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med. 2015;192(8):958-964. PubMed
15. Antonelli M, DeBacker D, Dorman T, Kleinpell R, Levy M, Rhodes A; Surviving Sepsis Campaign Executive Committee. Surviving Sepsis Campaign responds to Sepsis-3. Society of Critical Care Medicine website. http://www.survivingsepsis.org/SiteCollectionDocuments/SSC-Statements-Sepsis-Definitions-3-2016.pdf. Published March 1, 2016. Accessed May 11, 2016.

Issue
Journal of Hospital Medicine 12(4)
Issue
Journal of Hospital Medicine 12(4)
Page Number
256-258
Page Number
256-258
Publications
Publications
Topics
Article Type
Display Headline
Detecting sepsis: Are two opinions better than one?
Display Headline
Detecting sepsis: Are two opinions better than one?
Sections
Article Source

© 2017 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, MS, Section of Hospital Medicine, Department of Medicine, University of Chicago Medical Center, 5841 S Maryland Ave, W312, Chicago, IL 60637; Telephone: 773-834-2191; Fax: 773-795-7398; E-mail: dperes@medicine.bsd.uchicago.edu
Content Gating
Open Access (article Unlocked/Open Access)
Alternative CME
Use ProPublica
Article PDF Media
Media Files

ICU Transfer Delay and Outcome

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Association between intensive care unit transfer delay and hospital mortality: A multicenter investigation

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

Defining the Onset of Critical Illness

The eCART score, a statistically derived early warning score that is calculated based on patient demographic, vital sign, and laboratory data, was used as an objective measure of critical illness.[13] Score calculation was performed utilizing demographic information from administrative databases and time‐ and location‐stamped vital signs and laboratory results from data warehouses at the respective institutions. In this study, a score was calculated for each time‐stamped point in the entire dataset. Of note, eCART was not used in this population for patient care as this was a retrospective observational study. An eCART score at the 95% specificity cutoff for ICU transfer from the entire dataset defined a ward patient as critically ill, a definition created a priori and before any data analysis was performed.

Defining ICU Transfer Delay and Study Outcomes

The period of time from when a patient first reached this predefined eCART score to ICU transfer was calculated for each patient, up to a maximum of 24 hours. Transfer to the ICU greater than 6 hours after reaching the critical eCART score was defined a priori as a delayed transfer to allow comparisons between patients with nondelayed and delayed transfer. A patient who suffered a ward cardiac arrest with attempted resuscitation was counted as an ICU transfer at the time of arrest. If a patient experienced more than 1 ICU transfer during the admission, then only the first ward to ICU transfer was used. The primary outcome of the study was in‐hospital mortality, and secondary outcomes were ICU mortality and hospital LOS.

Statistical Analysis

Patient characteristics were compared between patients who experienced delayed and nondelayed ICU transfers using t tests, Wilcoxon rank sums, and [2] tests, as appropriate. The association between length of transfer delay and in‐hospital mortality was calculated using logistic regression, with adjustment for age, sex, and surgical status. In a post hoc sensitivity analysis, additional adjustments were made using each patient's first eCART score on the ward, the individual vital signs and laboratory variables from eCART, and whether the ICU transfer was due to a cardiac arrest on the wards. In addition, an interaction term between time to transfer and the initial eCART on the ward was added to determine if the association between delay and mortality varied by baseline severity. The change in eCART score over time was plotted from 12 hours before the time of first reaching the critical value until ICU transfer for those in the delayed and nondelayed groups using restricted cubic splines to compare the trajectories of severity of illness between these 2 groups. In addition, a linear regression model was fit to investigate the association between the eCART slope in the 8 hours prior to the critical eCART value until ICU transfer and the timing of ICU transfer delay. Statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX), and all tests of significance used a 2‐sided P<0.05.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Comparisons of Patient Characteristics Among All ICU Transfer Patients and Nondelayed (Within Six Hours) and Delayed Transfers Who Reached the Critical CART Score
Characteristic Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Data shown are mean (standard deviation) unless otherwise noted; n refers to the number of patients in each group. Abbreviations: eCART, electronic Cardiac Arrest Risk Triage; ICU, intensive care unit; IQR, interquartile range; LOS, length of stay. *For patients who survived to hospital discharge

Age, median (IQR), y 71 (5882) 73 (6083) 0.002
Female sex, n (%) 1,018 (49.5) 847 (48.8) 0.67
Race, n (%) 0.72
Black 467 (22.7) 374 (21.6)
White 1,141 (55.5) 971 (56.0)
Other/unknown 447 (21.8) 389 (22.4)
Surgical patient, n (%) 572 (27.8) 438 (25.2) 0.07
Hospital LOS prior to first critical eCART, median (IQR), d 1.5 (0.33.7) 1.6 (0.43.9) 0.04
Total hospital LOS, median (IQR), d* 11 (719) 13 (821) <0.001
Died during admission, n (%) 503 (24.5) 576 (33.2) <0.001
Comparison of Physiologic Variables at The time of ICU Transfer Between Nondelayed and Delayed ICU Transfers
Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Abbreviations: Alk phos, alkaline phosphatase; BUN, blood urea nitrogen; Cr, creatinine; eCART, electronic Cardiac Arrest Risk Triage; GFR, glomerular filtration rate; ICU, intensive care unit; IQR, interquartile range; K+, potassium; SGOT, serum glutamic‐oxaloacetic transaminase; SGPT, serum glutamic pyruvic transaminase; WBC, white blood cells.

  • All data are median (IQR) unless otherwise noted.

Respiratory rate, breaths/min 23 (1830) 22 (1828) <0.001
Systolic blood pressure, mm Hg 111 (92134) 109 (92128) 0.002
Diastolic blood pressure, mm Hg 61 (5075) 59 (4971) <0.001
Heart rate, beats/min 106 (88124) 101 (85117) <0.001
Oxygen saturation, median (IQR), % 97 (9499) 97 (9599) 0.15
Temperature, F 98.0 (97.299.1) 98.0 (97.199.0) 0.001
Alert mental status, number of observations (%) 1,749 (85%) 1,431 (83%) <0.001
eCART score at time of ICU transfer 61 (26122) 48 (21121) 0.914
WBC 10.3 (7.514.5) 11.7 (8.117.0) <0.001
Hemoglobin 10.7 (9.312.0) 10.3 (9.111.6) <0.001
Platelet 215 (137275) 195 (120269) 0.017
Sodium 137 (134140) 137 (134141) 0.70
K+ 4.1 (3.84.6) 4.2 (3.84.7) 0.006
Anion Gap 10 (813) 10 (814) <0.001
CO2 24 (2026) 23 (1826) <0.001
BUN 24 (1640) 32 (1853) <0.001
Cr 1.2 (0.92.0) 1.5 (1.02.7) <0.001
GFR 70 (7070) 70 (5170) <0.001
Glucose 123 (106161) 129 (105164) 0.48
Calcium 8.5 (7.98.8) 8.2 (7.78.7) <0.001
SGOT 26 (2635) 26 (2644) 0.001
SGPT 21 (2127) 21 (2033) 0.002
Total bilirubin 0.7 (0.71.0) 0.7 (0.71.3) <0.001
Alk phos 80 (8096) 80 (79111) 0.175
Albumin 3.0 (2.73.0) 3.0 (2.43.0) <0.001

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1
Association between length of intensive care unit (ICU) transfer delay and hospital mortality. Abbreviations: CI, confidence interval; eCART, electronic Cardiac Arrest Risk Triage.
Figure 2
Change in electronic Cardiac Arrest Risk Triage (eCART) score over time for the 12 hours prior to reaching the critical eCART value until intensive care unit (ICU) transfer for patients with delayed versus nondelayed ICU transfer. Time 0 denotes first critical eCART value.

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Acknowledgements

The authors thank Timothy Holper, Justin Lakeman, and Contessa Hsu for assistance with data extraction and technical support; Poome Chamnankit, MS, CNP, Kelly Bhatia, MSN, ACNP, and Audrey Seitman, MSN, ACNP for performing manual chart review of cardiac arrest patients; and Nicole Twu for administrative support.

Disclosures: This research was funded in part by an institutional Clinical and Translational Science Award grant (UL1 RR024999, PI: Dr. Julian Solway). Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Drs. Churpek and Wendlandt had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Preliminary versions of these data were presented at the 2015 meeting of the Society of Hospital Medicine (March 31, 2015, National Harbor, MD).

Files
References
  1. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  2. McQuillan P, Pilkington S, Allan A, et al. Confidential inquiry into quality of care before admission to intensive care. BMJ. 1998;316(7148):18531858.
  3. Town JA, Churpek MM, Yuen TC, Huber MT, Kress JP, Edelson DP. Relationship between ICU bed availability, ICU readmission, and cardiac arrest in the general wards. Crit Care Med. 2014;42(9):20372041.
  4. Simchen E, Sprung CL, Galai N, et al. Survival of critically ill patients hospitalized in and out of intensive care units under paucity of intensive care unit beds. Crit Care Med. 2004;32(8):16541661.
  5. Robert R, Reignier J, Tournoux‐Facon C, et al. Refusal of intensive care unit admission due to a full unit: impact on mortality. Am J Respir Crit Care Med. 2012;185(10):10811087.
  6. Sprung CL, Geber D, Eidelman LA, et al. Evaluation of triage decisions for intensive care admission. Crit Care Med. 1999;27(6):10731079.
  7. Garrouste‐Orgeas M, Montuclard L, Timsit JF, et al. Predictors of intensive care unit refusal in French intensive care units: a multiple‐center study. Crit Care Med. 2005;33(4):750755.
  8. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34(5):12971310.
  9. Cardoso LT, Grion CM, Matsuo T, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care. 2011;15(1):R28.
  10. Iapichino G, Corbella D, Minelli C, et al. Reasons for refusal of admission to intensive care and impact on mortality. Intensive Care Med. 2010;36(10):17721779.
  11. Hodgetts TJ, Kenward G, Vlackonikolis I, et al. Incidence, location and reasons for avoidable in‐hospital cardiac arrest in a district general hospital. Resuscitation. 2002;54(2):115123.
  12. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
  13. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  14. Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):13681377.
  15. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580637.
  16. Vanhecke TE, Gandhi M, McCullough PA, et al. Outcomes of patients considered for, but not admitted to, the intensive care unit. Crit Care Med. 2008;36(3):812817.
  17. Metcalfe MA, Sloggett A, McPherson K. Mortality among appropriately referred patients refused admission to intensive‐care units. Lancet. 1997;350(9070):711.
  18. Churpek MM, Yuen TC, Winslow C, Hall J, Edelson DP. Differences in vital signs between elderly and nonelderly patients prior to ward cardiac arrest. Crit Care Med. 2015;43(4):816822.
  19. Kang MA, Churpek MM, Zadravecz FJ, Adhikari R, Twu NM, Edelson DP. Real‐time risk prediction on the wards: a feasibility study [published April 13, 2016]. Crit Care Med. doi: 10.1097/CCM.0000000000001716.
  20. Smith GB, Prytherch DR, Schmidt PE, et al. Should age be included as a component of track and trigger systems used to identify sick adult patients? Resuscitation. 2008;78(2):109115.
  21. Duckitt RW, Buxton‐Thomas R, Walker J, et al. Worthing physiological scoring system: derivation and validation of a physiological early‐warning system for medical admissions. An observational, population‐based single‐centre study. Br J Anaesth. 2007;98(6):769774.
  22. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  23. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
Article PDF
Issue
Journal of Hospital Medicine - 11(11)
Publications
Page Number
757-762
Sections
Files
Files
Article PDF
Article PDF

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

Defining the Onset of Critical Illness

The eCART score, a statistically derived early warning score that is calculated based on patient demographic, vital sign, and laboratory data, was used as an objective measure of critical illness.[13] Score calculation was performed utilizing demographic information from administrative databases and time‐ and location‐stamped vital signs and laboratory results from data warehouses at the respective institutions. In this study, a score was calculated for each time‐stamped point in the entire dataset. Of note, eCART was not used in this population for patient care as this was a retrospective observational study. An eCART score at the 95% specificity cutoff for ICU transfer from the entire dataset defined a ward patient as critically ill, a definition created a priori and before any data analysis was performed.

Defining ICU Transfer Delay and Study Outcomes

The period of time from when a patient first reached this predefined eCART score to ICU transfer was calculated for each patient, up to a maximum of 24 hours. Transfer to the ICU greater than 6 hours after reaching the critical eCART score was defined a priori as a delayed transfer to allow comparisons between patients with nondelayed and delayed transfer. A patient who suffered a ward cardiac arrest with attempted resuscitation was counted as an ICU transfer at the time of arrest. If a patient experienced more than 1 ICU transfer during the admission, then only the first ward to ICU transfer was used. The primary outcome of the study was in‐hospital mortality, and secondary outcomes were ICU mortality and hospital LOS.

Statistical Analysis

Patient characteristics were compared between patients who experienced delayed and nondelayed ICU transfers using t tests, Wilcoxon rank sums, and [2] tests, as appropriate. The association between length of transfer delay and in‐hospital mortality was calculated using logistic regression, with adjustment for age, sex, and surgical status. In a post hoc sensitivity analysis, additional adjustments were made using each patient's first eCART score on the ward, the individual vital signs and laboratory variables from eCART, and whether the ICU transfer was due to a cardiac arrest on the wards. In addition, an interaction term between time to transfer and the initial eCART on the ward was added to determine if the association between delay and mortality varied by baseline severity. The change in eCART score over time was plotted from 12 hours before the time of first reaching the critical value until ICU transfer for those in the delayed and nondelayed groups using restricted cubic splines to compare the trajectories of severity of illness between these 2 groups. In addition, a linear regression model was fit to investigate the association between the eCART slope in the 8 hours prior to the critical eCART value until ICU transfer and the timing of ICU transfer delay. Statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX), and all tests of significance used a 2‐sided P<0.05.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Comparisons of Patient Characteristics Among All ICU Transfer Patients and Nondelayed (Within Six Hours) and Delayed Transfers Who Reached the Critical CART Score
Characteristic Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Data shown are mean (standard deviation) unless otherwise noted; n refers to the number of patients in each group. Abbreviations: eCART, electronic Cardiac Arrest Risk Triage; ICU, intensive care unit; IQR, interquartile range; LOS, length of stay. *For patients who survived to hospital discharge

Age, median (IQR), y 71 (5882) 73 (6083) 0.002
Female sex, n (%) 1,018 (49.5) 847 (48.8) 0.67
Race, n (%) 0.72
Black 467 (22.7) 374 (21.6)
White 1,141 (55.5) 971 (56.0)
Other/unknown 447 (21.8) 389 (22.4)
Surgical patient, n (%) 572 (27.8) 438 (25.2) 0.07
Hospital LOS prior to first critical eCART, median (IQR), d 1.5 (0.33.7) 1.6 (0.43.9) 0.04
Total hospital LOS, median (IQR), d* 11 (719) 13 (821) <0.001
Died during admission, n (%) 503 (24.5) 576 (33.2) <0.001
Comparison of Physiologic Variables at The time of ICU Transfer Between Nondelayed and Delayed ICU Transfers
Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Abbreviations: Alk phos, alkaline phosphatase; BUN, blood urea nitrogen; Cr, creatinine; eCART, electronic Cardiac Arrest Risk Triage; GFR, glomerular filtration rate; ICU, intensive care unit; IQR, interquartile range; K+, potassium; SGOT, serum glutamic‐oxaloacetic transaminase; SGPT, serum glutamic pyruvic transaminase; WBC, white blood cells.

  • All data are median (IQR) unless otherwise noted.

Respiratory rate, breaths/min 23 (1830) 22 (1828) <0.001
Systolic blood pressure, mm Hg 111 (92134) 109 (92128) 0.002
Diastolic blood pressure, mm Hg 61 (5075) 59 (4971) <0.001
Heart rate, beats/min 106 (88124) 101 (85117) <0.001
Oxygen saturation, median (IQR), % 97 (9499) 97 (9599) 0.15
Temperature, F 98.0 (97.299.1) 98.0 (97.199.0) 0.001
Alert mental status, number of observations (%) 1,749 (85%) 1,431 (83%) <0.001
eCART score at time of ICU transfer 61 (26122) 48 (21121) 0.914
WBC 10.3 (7.514.5) 11.7 (8.117.0) <0.001
Hemoglobin 10.7 (9.312.0) 10.3 (9.111.6) <0.001
Platelet 215 (137275) 195 (120269) 0.017
Sodium 137 (134140) 137 (134141) 0.70
K+ 4.1 (3.84.6) 4.2 (3.84.7) 0.006
Anion Gap 10 (813) 10 (814) <0.001
CO2 24 (2026) 23 (1826) <0.001
BUN 24 (1640) 32 (1853) <0.001
Cr 1.2 (0.92.0) 1.5 (1.02.7) <0.001
GFR 70 (7070) 70 (5170) <0.001
Glucose 123 (106161) 129 (105164) 0.48
Calcium 8.5 (7.98.8) 8.2 (7.78.7) <0.001
SGOT 26 (2635) 26 (2644) 0.001
SGPT 21 (2127) 21 (2033) 0.002
Total bilirubin 0.7 (0.71.0) 0.7 (0.71.3) <0.001
Alk phos 80 (8096) 80 (79111) 0.175
Albumin 3.0 (2.73.0) 3.0 (2.43.0) <0.001

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1
Association between length of intensive care unit (ICU) transfer delay and hospital mortality. Abbreviations: CI, confidence interval; eCART, electronic Cardiac Arrest Risk Triage.
Figure 2
Change in electronic Cardiac Arrest Risk Triage (eCART) score over time for the 12 hours prior to reaching the critical eCART value until intensive care unit (ICU) transfer for patients with delayed versus nondelayed ICU transfer. Time 0 denotes first critical eCART value.

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Acknowledgements

The authors thank Timothy Holper, Justin Lakeman, and Contessa Hsu for assistance with data extraction and technical support; Poome Chamnankit, MS, CNP, Kelly Bhatia, MSN, ACNP, and Audrey Seitman, MSN, ACNP for performing manual chart review of cardiac arrest patients; and Nicole Twu for administrative support.

Disclosures: This research was funded in part by an institutional Clinical and Translational Science Award grant (UL1 RR024999, PI: Dr. Julian Solway). Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Drs. Churpek and Wendlandt had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Preliminary versions of these data were presented at the 2015 meeting of the Society of Hospital Medicine (March 31, 2015, National Harbor, MD).

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

Defining the Onset of Critical Illness

The eCART score, a statistically derived early warning score that is calculated based on patient demographic, vital sign, and laboratory data, was used as an objective measure of critical illness.[13] Score calculation was performed utilizing demographic information from administrative databases and time‐ and location‐stamped vital signs and laboratory results from data warehouses at the respective institutions. In this study, a score was calculated for each time‐stamped point in the entire dataset. Of note, eCART was not used in this population for patient care as this was a retrospective observational study. An eCART score at the 95% specificity cutoff for ICU transfer from the entire dataset defined a ward patient as critically ill, a definition created a priori and before any data analysis was performed.

Defining ICU Transfer Delay and Study Outcomes

The period of time from when a patient first reached this predefined eCART score to ICU transfer was calculated for each patient, up to a maximum of 24 hours. Transfer to the ICU greater than 6 hours after reaching the critical eCART score was defined a priori as a delayed transfer to allow comparisons between patients with nondelayed and delayed transfer. A patient who suffered a ward cardiac arrest with attempted resuscitation was counted as an ICU transfer at the time of arrest. If a patient experienced more than 1 ICU transfer during the admission, then only the first ward to ICU transfer was used. The primary outcome of the study was in‐hospital mortality, and secondary outcomes were ICU mortality and hospital LOS.

Statistical Analysis

Patient characteristics were compared between patients who experienced delayed and nondelayed ICU transfers using t tests, Wilcoxon rank sums, and [2] tests, as appropriate. The association between length of transfer delay and in‐hospital mortality was calculated using logistic regression, with adjustment for age, sex, and surgical status. In a post hoc sensitivity analysis, additional adjustments were made using each patient's first eCART score on the ward, the individual vital signs and laboratory variables from eCART, and whether the ICU transfer was due to a cardiac arrest on the wards. In addition, an interaction term between time to transfer and the initial eCART on the ward was added to determine if the association between delay and mortality varied by baseline severity. The change in eCART score over time was plotted from 12 hours before the time of first reaching the critical value until ICU transfer for those in the delayed and nondelayed groups using restricted cubic splines to compare the trajectories of severity of illness between these 2 groups. In addition, a linear regression model was fit to investigate the association between the eCART slope in the 8 hours prior to the critical eCART value until ICU transfer and the timing of ICU transfer delay. Statistical analyses were performed using Stata version 12.1 (StataCorp, College Station, TX), and all tests of significance used a 2‐sided P<0.05.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Comparisons of Patient Characteristics Among All ICU Transfer Patients and Nondelayed (Within Six Hours) and Delayed Transfers Who Reached the Critical CART Score
Characteristic Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Data shown are mean (standard deviation) unless otherwise noted; n refers to the number of patients in each group. Abbreviations: eCART, electronic Cardiac Arrest Risk Triage; ICU, intensive care unit; IQR, interquartile range; LOS, length of stay. *For patients who survived to hospital discharge

Age, median (IQR), y 71 (5882) 73 (6083) 0.002
Female sex, n (%) 1,018 (49.5) 847 (48.8) 0.67
Race, n (%) 0.72
Black 467 (22.7) 374 (21.6)
White 1,141 (55.5) 971 (56.0)
Other/unknown 447 (21.8) 389 (22.4)
Surgical patient, n (%) 572 (27.8) 438 (25.2) 0.07
Hospital LOS prior to first critical eCART, median (IQR), d 1.5 (0.33.7) 1.6 (0.43.9) 0.04
Total hospital LOS, median (IQR), d* 11 (719) 13 (821) <0.001
Died during admission, n (%) 503 (24.5) 576 (33.2) <0.001
Comparison of Physiologic Variables at The time of ICU Transfer Between Nondelayed and Delayed ICU Transfers
Transferred Within 6 Hours, n=2,055 Transfer Delayed, n=1,734 P Value
  • NOTE: Abbreviations: Alk phos, alkaline phosphatase; BUN, blood urea nitrogen; Cr, creatinine; eCART, electronic Cardiac Arrest Risk Triage; GFR, glomerular filtration rate; ICU, intensive care unit; IQR, interquartile range; K+, potassium; SGOT, serum glutamic‐oxaloacetic transaminase; SGPT, serum glutamic pyruvic transaminase; WBC, white blood cells.

  • All data are median (IQR) unless otherwise noted.

Respiratory rate, breaths/min 23 (1830) 22 (1828) <0.001
Systolic blood pressure, mm Hg 111 (92134) 109 (92128) 0.002
Diastolic blood pressure, mm Hg 61 (5075) 59 (4971) <0.001
Heart rate, beats/min 106 (88124) 101 (85117) <0.001
Oxygen saturation, median (IQR), % 97 (9499) 97 (9599) 0.15
Temperature, F 98.0 (97.299.1) 98.0 (97.199.0) 0.001
Alert mental status, number of observations (%) 1,749 (85%) 1,431 (83%) <0.001
eCART score at time of ICU transfer 61 (26122) 48 (21121) 0.914
WBC 10.3 (7.514.5) 11.7 (8.117.0) <0.001
Hemoglobin 10.7 (9.312.0) 10.3 (9.111.6) <0.001
Platelet 215 (137275) 195 (120269) 0.017
Sodium 137 (134140) 137 (134141) 0.70
K+ 4.1 (3.84.6) 4.2 (3.84.7) 0.006
Anion Gap 10 (813) 10 (814) <0.001
CO2 24 (2026) 23 (1826) <0.001
BUN 24 (1640) 32 (1853) <0.001
Cr 1.2 (0.92.0) 1.5 (1.02.7) <0.001
GFR 70 (7070) 70 (5170) <0.001
Glucose 123 (106161) 129 (105164) 0.48
Calcium 8.5 (7.98.8) 8.2 (7.78.7) <0.001
SGOT 26 (2635) 26 (2644) 0.001
SGPT 21 (2127) 21 (2033) 0.002
Total bilirubin 0.7 (0.71.0) 0.7 (0.71.3) <0.001
Alk phos 80 (8096) 80 (79111) 0.175
Albumin 3.0 (2.73.0) 3.0 (2.43.0) <0.001

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1
Association between length of intensive care unit (ICU) transfer delay and hospital mortality. Abbreviations: CI, confidence interval; eCART, electronic Cardiac Arrest Risk Triage.
Figure 2
Change in electronic Cardiac Arrest Risk Triage (eCART) score over time for the 12 hours prior to reaching the critical eCART value until intensive care unit (ICU) transfer for patients with delayed versus nondelayed ICU transfer. Time 0 denotes first critical eCART value.

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Acknowledgements

The authors thank Timothy Holper, Justin Lakeman, and Contessa Hsu for assistance with data extraction and technical support; Poome Chamnankit, MS, CNP, Kelly Bhatia, MSN, ACNP, and Audrey Seitman, MSN, ACNP for performing manual chart review of cardiac arrest patients; and Nicole Twu for administrative support.

Disclosures: This research was funded in part by an institutional Clinical and Translational Science Award grant (UL1 RR024999, PI: Dr. Julian Solway). Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Drs. Churpek and Wendlandt had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Preliminary versions of these data were presented at the 2015 meeting of the Society of Hospital Medicine (March 31, 2015, National Harbor, MD).

References
  1. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  2. McQuillan P, Pilkington S, Allan A, et al. Confidential inquiry into quality of care before admission to intensive care. BMJ. 1998;316(7148):18531858.
  3. Town JA, Churpek MM, Yuen TC, Huber MT, Kress JP, Edelson DP. Relationship between ICU bed availability, ICU readmission, and cardiac arrest in the general wards. Crit Care Med. 2014;42(9):20372041.
  4. Simchen E, Sprung CL, Galai N, et al. Survival of critically ill patients hospitalized in and out of intensive care units under paucity of intensive care unit beds. Crit Care Med. 2004;32(8):16541661.
  5. Robert R, Reignier J, Tournoux‐Facon C, et al. Refusal of intensive care unit admission due to a full unit: impact on mortality. Am J Respir Crit Care Med. 2012;185(10):10811087.
  6. Sprung CL, Geber D, Eidelman LA, et al. Evaluation of triage decisions for intensive care admission. Crit Care Med. 1999;27(6):10731079.
  7. Garrouste‐Orgeas M, Montuclard L, Timsit JF, et al. Predictors of intensive care unit refusal in French intensive care units: a multiple‐center study. Crit Care Med. 2005;33(4):750755.
  8. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34(5):12971310.
  9. Cardoso LT, Grion CM, Matsuo T, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care. 2011;15(1):R28.
  10. Iapichino G, Corbella D, Minelli C, et al. Reasons for refusal of admission to intensive care and impact on mortality. Intensive Care Med. 2010;36(10):17721779.
  11. Hodgetts TJ, Kenward G, Vlackonikolis I, et al. Incidence, location and reasons for avoidable in‐hospital cardiac arrest in a district general hospital. Resuscitation. 2002;54(2):115123.
  12. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
  13. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  14. Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):13681377.
  15. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580637.
  16. Vanhecke TE, Gandhi M, McCullough PA, et al. Outcomes of patients considered for, but not admitted to, the intensive care unit. Crit Care Med. 2008;36(3):812817.
  17. Metcalfe MA, Sloggett A, McPherson K. Mortality among appropriately referred patients refused admission to intensive‐care units. Lancet. 1997;350(9070):711.
  18. Churpek MM, Yuen TC, Winslow C, Hall J, Edelson DP. Differences in vital signs between elderly and nonelderly patients prior to ward cardiac arrest. Crit Care Med. 2015;43(4):816822.
  19. Kang MA, Churpek MM, Zadravecz FJ, Adhikari R, Twu NM, Edelson DP. Real‐time risk prediction on the wards: a feasibility study [published April 13, 2016]. Crit Care Med. doi: 10.1097/CCM.0000000000001716.
  20. Smith GB, Prytherch DR, Schmidt PE, et al. Should age be included as a component of track and trigger systems used to identify sick adult patients? Resuscitation. 2008;78(2):109115.
  21. Duckitt RW, Buxton‐Thomas R, Walker J, et al. Worthing physiological scoring system: derivation and validation of a physiological early‐warning system for medical admissions. An observational, population‐based single‐centre study. Br J Anaesth. 2007;98(6):769774.
  22. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  23. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
References
  1. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  2. McQuillan P, Pilkington S, Allan A, et al. Confidential inquiry into quality of care before admission to intensive care. BMJ. 1998;316(7148):18531858.
  3. Town JA, Churpek MM, Yuen TC, Huber MT, Kress JP, Edelson DP. Relationship between ICU bed availability, ICU readmission, and cardiac arrest in the general wards. Crit Care Med. 2014;42(9):20372041.
  4. Simchen E, Sprung CL, Galai N, et al. Survival of critically ill patients hospitalized in and out of intensive care units under paucity of intensive care unit beds. Crit Care Med. 2004;32(8):16541661.
  5. Robert R, Reignier J, Tournoux‐Facon C, et al. Refusal of intensive care unit admission due to a full unit: impact on mortality. Am J Respir Crit Care Med. 2012;185(10):10811087.
  6. Sprung CL, Geber D, Eidelman LA, et al. Evaluation of triage decisions for intensive care admission. Crit Care Med. 1999;27(6):10731079.
  7. Garrouste‐Orgeas M, Montuclard L, Timsit JF, et al. Predictors of intensive care unit refusal in French intensive care units: a multiple‐center study. Crit Care Med. 2005;33(4):750755.
  8. Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today's critically ill patients. Crit Care Med. 2006;34(5):12971310.
  9. Cardoso LT, Grion CM, Matsuo T, et al. Impact of delayed admission to intensive care units on mortality of critically ill patients: a cohort study. Crit Care. 2011;15(1):R28.
  10. Iapichino G, Corbella D, Minelli C, et al. Reasons for refusal of admission to intensive care and impact on mortality. Intensive Care Med. 2010;36(10):17721779.
  11. Hodgetts TJ, Kenward G, Vlackonikolis I, et al. Incidence, location and reasons for avoidable in‐hospital cardiac arrest in a district general hospital. Resuscitation. 2002;54(2):115123.
  12. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
  13. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  14. Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):13681377.
  15. Dellinger RP, Levy MM, Rhodes A, et al. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580637.
  16. Vanhecke TE, Gandhi M, McCullough PA, et al. Outcomes of patients considered for, but not admitted to, the intensive care unit. Crit Care Med. 2008;36(3):812817.
  17. Metcalfe MA, Sloggett A, McPherson K. Mortality among appropriately referred patients refused admission to intensive‐care units. Lancet. 1997;350(9070):711.
  18. Churpek MM, Yuen TC, Winslow C, Hall J, Edelson DP. Differences in vital signs between elderly and nonelderly patients prior to ward cardiac arrest. Crit Care Med. 2015;43(4):816822.
  19. Kang MA, Churpek MM, Zadravecz FJ, Adhikari R, Twu NM, Edelson DP. Real‐time risk prediction on the wards: a feasibility study [published April 13, 2016]. Crit Care Med. doi: 10.1097/CCM.0000000000001716.
  20. Smith GB, Prytherch DR, Schmidt PE, et al. Should age be included as a component of track and trigger systems used to identify sick adult patients? Resuscitation. 2008;78(2):109115.
  21. Duckitt RW, Buxton‐Thomas R, Walker J, et al. Worthing physiological scoring system: derivation and validation of a physiological early‐warning system for medical admissions. An observational, population‐based single‐centre study. Br J Anaesth. 2007;98(6):769774.
  22. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  23. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
Issue
Journal of Hospital Medicine - 11(11)
Issue
Journal of Hospital Medicine - 11(11)
Page Number
757-762
Page Number
757-762
Publications
Publications
Article Type
Display Headline
Association between intensive care unit transfer delay and hospital mortality: A multicenter investigation
Display Headline
Association between intensive care unit transfer delay and hospital mortality: A multicenter investigation
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Matthew M. Churpek, MD, University of Chicago Medical Center, Section of Pulmonary and Critical Care Medicine, 5841 South Maryland Avenue, MC 6076, Chicago, IL 60637; Telephone: 773‐702‐1092; Fax: 773‐702‐6500; E‐mail: matthew.churpek@uchospitals.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Electronic Assessment of Mental Status

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

Files
References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
Article PDF
Issue
Journal of Hospital Medicine - 11(7)
Publications
Page Number
463-466
Sections
Files
Files
Article PDF
Article PDF

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
Issue
Journal of Hospital Medicine - 11(7)
Issue
Journal of Hospital Medicine - 11(7)
Page Number
463-466
Page Number
463-466
Publications
Publications
Article Type
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, University of Chicago Medicine, Section of Hospital Medicine, 5841 S. Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773‐834‐2191; Fax: 773‐795‐7398; E‐mail: dperes@medicine.bsd.uchicago.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files

Mental Status to Predict Mortality

Article Type
Changed
Tue, 05/16/2017 - 23:02
Display Headline
Comparison of mental‐status scales for predicting mortality on the general wards

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

Study Setting and Protocol

We conducted an observational cohort study of consecutive adult general‐ward admissions from July 2011 through January 2013 at a 500‐bed, urban US teaching hospital. During the study period, no early‐warning scoring systems were in place on the hospital wards. Rapid response teams responding to altered mental status would do so without specific thresholds for activation. During this period, nurses on the general floors were expected to record each patient's GCS and RASS score in the electronic health record (EPIC Systems Corp., Verona, WI) as part of the routine patient assessment at least once every 12‐hour shift. AVPU assessments were extracted from the eye component of the GCS. The letter A was assigned to a GCS Eye score of 4 (opens eyes spontaneously), V to a score of 3 (opens eyes in response to voice), P to a score of 2 (opens eyes in response to painful stimuli), and U to a score of 1 (does not open eyes). To avoid comparison of mental‐status scores at different time points, only concurrent GCS and RASS scores, documented within 10 minutes of one another, were included in the analysis.

Location and time‐stamped GCS and RASS scores, demographics, and in‐hospital mortality data were obtained from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol and data‐collection mechanisms were approved by The University of Chicago Institutional Review Board (#16995A).

Statistical Analysis

Baseline admission characteristics were described using proportions (%) and measures of central tendency (mean, standard deviations [SD]; median, interquartile ranges [IQR]). Patient severity of illness at first ward observation was calculated using the MEWS.[19] All mental‐status observations during a patient's ward stay were included in the analysis. Odds ratios for 24‐hour mortality following an abnormal mental‐status score were calculated using generalized estimating equations, with an exchangeable correlation structure to account for the correlation of scores within the same patient, as more than 1 abnormal mental‐status score may have been documented within the 24 hours preceding death. Spearman's rank correlation coefficients () were used to estimate the correlation among AVPU, GCS, and RASS scores.

The predictive accuracies of AVPU, GCS, RASS, and the subscales of GCS were compared using the area under the receiver operating characteristic curve (AUC), with mortality within 24 hours of a mental‐status observation as the primary outcome and the mental‐status score as the predictor variable. Although AUCs are typically used as a measure of discriminative ability, this study used AUCs to summarize both sensitivity and specificity across a range of cutoffs, providing an overall measure of predictive accuracies across mental‐status scales. To estimate AUCs, the AVPU, GCS, and GCS subscales were entered into a logistic regression model as ordinal variables, whereas RASS was entered as a nominal variable due to its positive and negative components, and predicted probabilities were calculated. In addition, a combined model was fit where GCS and RASS were classified as categorical independent variables. AUCs were then calculated by utilizing the predicted probabilities from each logistic regression model using the trapezoidal rule.[20] A sensitivity analysis was performed to estimate the internal validity of the RASS model using 10‐fold cross‐validation.

Predefined subgroup analyses were performed that compared the accuracies of AVPU, GCS, and RASS for predicting 24‐hour mortality in patients above and below the median age of the study population, and between patients who underwent surgery during their admission or not (surgical vs medical). All tests of significance used a 2‐sided P value <0.05. All data analysis was performed using Stata version 13.0 (StataCorp, College Station, TX).

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

Baseline Characteristics of Hospital Admissions
  • NOTE: Characteristics are stratified at the hospital admission level. Abbreviations: IQR, interquartile range; MEWS, Modified Early Warning Score; n, number of observations; SD, standard deviation.

Total no. of admissions26,873
Total no. of unique patients17,660
Age, y, mean (SD)57 (17)
Female sex, n (%)14,293 (53)
Race, n (%) 
White10,516 (39)
Black12,580 (47)
Other/unknown3,777 (14)
Admission MEWS, median (IQR)1 (11)
Days on ward, median (IQR)5 (310)
Observations per person, per day, median (IQR)3 (24)
Underwent surgery during hospitalization, n (%)6,141 (23)
Deaths, n (%)417 (1.6)

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1
Score correlations between (1a) AVPU and GCS total, and between (1b) AVPU and RASS. Boxes indicate interquartile range (25th to 75th percentiles), whiskers indicate 5th to 95th percentiles, and diamonds indicate median. Each correlation is significant at P < 0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Odds of Mortality Within 24 Hours of an Abnormal Mental‐Status Score
Mental‐status ScoreObservations, n (%)Odds Ratio for Mortality (95% CI)
  • NOTE: Odds ratios, with 95% CIs, comparing the probability of mortality within 24 hours of an abnormal mental‐status score to the probability of mortality within 24 hours of a normal mental‐status score (Reference). All calculations control for clustering of observations within the same admission. All odds ratios were significant at P<0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; CI, confidence interval; GCS, Glascow Coma Scale; n, number of observations; RASS, Richmond Agitation Sedation Scale.

GCS Eye (AVPU)  
4 (alert)289,857 (98)Reference
<4 (not alert)6,117 (2)33.8 (23.947.9)
GCS Verbal  
5277,862 (94)Reference
411,258 (4)4.7 (2.87.9)
<46,854 (2)52.7 (38.073.2)
GCS Motor  
6287,441 (97)Reference
<68,533 (3)41.8 (30.756.9)
GCS total  
15276,042 (93)Reference
13, 1412,437 (4)5.2 (3.38.3)
<137,495 (3)55.5 (40.077.1)
RASS  
>06,867 (2)8.5 (5.613.0)
0275,708 (93)Reference
<013,339 (5)25.8 (19.234.6)

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2
Predictive accuracies of mental‐status scales (and GCS subscales) for mortality within 24 hours of a mental‐status observation (*P < 0.001). AUC with whiskers indicating 95% confidence intervals for predicting mortality occurring within 24 hours of a mental‐status observation. AUCs are shown for each mental‐status scale, for the combination of GCS and RASS, and for the 3 subscales of the GCS. Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Figure 3
Predictive accuracies of AVPU, GCS, and RASS for mortality within 24 hours of a mental‐status observation. Subgroup analysis is based on age and surgical status (*P < 0.05, **P < 0.001). AUC with whiskers indicating 95% CI for predicting mortality occurring within 24 hours of a mental‐status observation, analyzed at the observation level, and stratified by patient age (below or greater than or equal to the median age of 57 years) and surgical status (patient with surgery during hospitalization or medical patient only). Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Disclosures

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Edelson has received research support from the National Heart, Lung, and Blood Institute (K23 HL097157), Philips (Andover, MA), the American Heart Association (Dallas, TX), Laerdal Medical (Stavanger, Norway), and Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no conflicts of interest.

Files
References
  1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):17531762.
  2. Pompei P, Foreman M, Rudberg MA, Inouye SK, Braund V, Cassel CK. Delirium in hospitalized older persons: outcomes and predictors. J Am Geriatr Soc. 1994;42(8):809815.
  3. Siddiqi N, House AO, Holmes JD. Occurrence and outcome of delirium in medical in‐patients: a systematic literature review. Age Ageing. 2006;35(4):350364.
  4. Levkoff SE, Evans DA, Liptzin B, et al. Delirium. The occurrence and persistence of symptoms among elderly hospitalized patients. Arch Intern Med. 1992;152(2):334340.
  5. Dyer CB, Ashton CM, Teasdale TA. Postoperative delirium. A review of 80 primary data‐collection studies. Arch Intern Med. 1995;155(5):461465.
  6. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM Nurses' recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):24672473.
  7. Armstrong SC, Cozza KL, Watanabe KS. The misdiagnosis of delirium. Psychosomatics. 1997;38(5):433439.
  8. Ely EW, Stephens RK, Jackson JC, et al. Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professionals. Crit Care Med. 2004;32(1):106112.
  9. Farrell KR, Ganzini L. Misdiagnosing delirium as depression in medically ill elderly patients. Arch Intern Med. 1995;155(22):24592464.
  10. Simpson CJ. Doctors and nurses use of the word confused. Br J Psychiatry. 1984;145:441443.
  11. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation‐Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):13381344.
  12. Teasdale G, Jennett B. Assessment and prognosis of coma after head injury. Acta Neurochir (Wien). 1976;34(1–4):4555.
  13. Bastos PG, Sun X, Wagner DP, Wu AW, Knaus WA. Glasgow Coma Scale score in the evaluation of outcome in the intensive care unit: findings from the Acute Physiology and Chronic Health Evaluation III study. Crit Care Med. 1993;21(10):14591465.
  14. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
  15. Menegazzi JJ, Davis EA, Sucov AN, Paris PM. Reliability of the Glasgow Coma Scale when used by emergency physicians and paramedics. J Trauma. 1993;34(1):4648.
  16. Alexander RH, Proctor HJ; American College of Surgeons. Committee on Trauma. Advanced Trauma Life Support Program For Physicians: ATLS. 5th ed. Chicago, IL: American College of Surgeons; 1993.
  17. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation. 2008;77(2):170179.
  18. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, Higgins B. A review, and performance evaluation, of single‐parameter “track and trigger” systems. Resuscitation. 2008;79(1):1121.
  19. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning score in medical admissions. QJM. 2001;94(10):521526.
  20. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  21. McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glascow Coma Scale. Anaesthesia. 2004;59(1):3437.
  22. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  23. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  24. Peterson JF, Pun BT, Dittus RS, et al. Delirium and its motoric subtypes: a study of 614 critically ill patients. J Am Geriatr Soc. 2006;54(3):479484.
  25. Angles EM, Robinson TN, Biffl WL, et al. Risk factors for delirium after major trauma. Am J Surg. 2008;196(6):864869.
  26. Meagher DJ, O'Hanlon D, O'Mahony E, Casey PR, Trzepacz PT. Relationship between symptoms and motoric subtype of delirium. J Neuropsychiatry Clin Neurosci. 2000;12(1):5156.
  27. Yang FM, Marcantonio ER, Inouye SK, et al. Phenomenological subtypes of delirium in older persons: patterns, prevalence, and prognosis. Psychosomatics. 2009;50(3):248254.
  28. Kiely DK, Jones RN, Bergmann MA, Marcantonio ER. Association between psychomotor activity delirium subtypes and mortality among newly admitted post‐acute facility patients. J Gerontol A Biol Sci Med Sci. 2007;62(2):174179.
  29. Marcantonio E, Ta T, Duthie E, Resnick NM. Delirium severity and psychomotor types: their relationship with outcomes after hip fracture repair. J Am Geriatr Soc. 2002;50(5):850857.
  30. Rowley G, Fielding K. Reliability and accuracy of the Glasgow Coma Scale with experienced and inexperienced users. Lancet. 1991;337(8740):535538.
  31. Pun BT, Gordon SM, Peterson JF, et al. Large‐scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med. 2005;33(6):11991205.
  32. Vasilevskis EE, Morandi A, Boehm L, et al. Delirium and sedation recognition using validated instruments: reliability of bedside intensive care unit nursing assessments from 2007 to 2010. J Am Geriatr Soc. 2011;59(suppl 2):S249S255.
Article PDF
Issue
Journal of Hospital Medicine - 10(10)
Publications
Page Number
658-663
Sections
Files
Files
Article PDF
Article PDF

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

Study Setting and Protocol

We conducted an observational cohort study of consecutive adult general‐ward admissions from July 2011 through January 2013 at a 500‐bed, urban US teaching hospital. During the study period, no early‐warning scoring systems were in place on the hospital wards. Rapid response teams responding to altered mental status would do so without specific thresholds for activation. During this period, nurses on the general floors were expected to record each patient's GCS and RASS score in the electronic health record (EPIC Systems Corp., Verona, WI) as part of the routine patient assessment at least once every 12‐hour shift. AVPU assessments were extracted from the eye component of the GCS. The letter A was assigned to a GCS Eye score of 4 (opens eyes spontaneously), V to a score of 3 (opens eyes in response to voice), P to a score of 2 (opens eyes in response to painful stimuli), and U to a score of 1 (does not open eyes). To avoid comparison of mental‐status scores at different time points, only concurrent GCS and RASS scores, documented within 10 minutes of one another, were included in the analysis.

Location and time‐stamped GCS and RASS scores, demographics, and in‐hospital mortality data were obtained from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol and data‐collection mechanisms were approved by The University of Chicago Institutional Review Board (#16995A).

Statistical Analysis

Baseline admission characteristics were described using proportions (%) and measures of central tendency (mean, standard deviations [SD]; median, interquartile ranges [IQR]). Patient severity of illness at first ward observation was calculated using the MEWS.[19] All mental‐status observations during a patient's ward stay were included in the analysis. Odds ratios for 24‐hour mortality following an abnormal mental‐status score were calculated using generalized estimating equations, with an exchangeable correlation structure to account for the correlation of scores within the same patient, as more than 1 abnormal mental‐status score may have been documented within the 24 hours preceding death. Spearman's rank correlation coefficients () were used to estimate the correlation among AVPU, GCS, and RASS scores.

The predictive accuracies of AVPU, GCS, RASS, and the subscales of GCS were compared using the area under the receiver operating characteristic curve (AUC), with mortality within 24 hours of a mental‐status observation as the primary outcome and the mental‐status score as the predictor variable. Although AUCs are typically used as a measure of discriminative ability, this study used AUCs to summarize both sensitivity and specificity across a range of cutoffs, providing an overall measure of predictive accuracies across mental‐status scales. To estimate AUCs, the AVPU, GCS, and GCS subscales were entered into a logistic regression model as ordinal variables, whereas RASS was entered as a nominal variable due to its positive and negative components, and predicted probabilities were calculated. In addition, a combined model was fit where GCS and RASS were classified as categorical independent variables. AUCs were then calculated by utilizing the predicted probabilities from each logistic regression model using the trapezoidal rule.[20] A sensitivity analysis was performed to estimate the internal validity of the RASS model using 10‐fold cross‐validation.

Predefined subgroup analyses were performed that compared the accuracies of AVPU, GCS, and RASS for predicting 24‐hour mortality in patients above and below the median age of the study population, and between patients who underwent surgery during their admission or not (surgical vs medical). All tests of significance used a 2‐sided P value <0.05. All data analysis was performed using Stata version 13.0 (StataCorp, College Station, TX).

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

Baseline Characteristics of Hospital Admissions
  • NOTE: Characteristics are stratified at the hospital admission level. Abbreviations: IQR, interquartile range; MEWS, Modified Early Warning Score; n, number of observations; SD, standard deviation.

Total no. of admissions26,873
Total no. of unique patients17,660
Age, y, mean (SD)57 (17)
Female sex, n (%)14,293 (53)
Race, n (%) 
White10,516 (39)
Black12,580 (47)
Other/unknown3,777 (14)
Admission MEWS, median (IQR)1 (11)
Days on ward, median (IQR)5 (310)
Observations per person, per day, median (IQR)3 (24)
Underwent surgery during hospitalization, n (%)6,141 (23)
Deaths, n (%)417 (1.6)

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1
Score correlations between (1a) AVPU and GCS total, and between (1b) AVPU and RASS. Boxes indicate interquartile range (25th to 75th percentiles), whiskers indicate 5th to 95th percentiles, and diamonds indicate median. Each correlation is significant at P < 0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Odds of Mortality Within 24 Hours of an Abnormal Mental‐Status Score
Mental‐status ScoreObservations, n (%)Odds Ratio for Mortality (95% CI)
  • NOTE: Odds ratios, with 95% CIs, comparing the probability of mortality within 24 hours of an abnormal mental‐status score to the probability of mortality within 24 hours of a normal mental‐status score (Reference). All calculations control for clustering of observations within the same admission. All odds ratios were significant at P<0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; CI, confidence interval; GCS, Glascow Coma Scale; n, number of observations; RASS, Richmond Agitation Sedation Scale.

GCS Eye (AVPU)  
4 (alert)289,857 (98)Reference
<4 (not alert)6,117 (2)33.8 (23.947.9)
GCS Verbal  
5277,862 (94)Reference
411,258 (4)4.7 (2.87.9)
<46,854 (2)52.7 (38.073.2)
GCS Motor  
6287,441 (97)Reference
<68,533 (3)41.8 (30.756.9)
GCS total  
15276,042 (93)Reference
13, 1412,437 (4)5.2 (3.38.3)
<137,495 (3)55.5 (40.077.1)
RASS  
>06,867 (2)8.5 (5.613.0)
0275,708 (93)Reference
<013,339 (5)25.8 (19.234.6)

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2
Predictive accuracies of mental‐status scales (and GCS subscales) for mortality within 24 hours of a mental‐status observation (*P < 0.001). AUC with whiskers indicating 95% confidence intervals for predicting mortality occurring within 24 hours of a mental‐status observation. AUCs are shown for each mental‐status scale, for the combination of GCS and RASS, and for the 3 subscales of the GCS. Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Figure 3
Predictive accuracies of AVPU, GCS, and RASS for mortality within 24 hours of a mental‐status observation. Subgroup analysis is based on age and surgical status (*P < 0.05, **P < 0.001). AUC with whiskers indicating 95% CI for predicting mortality occurring within 24 hours of a mental‐status observation, analyzed at the observation level, and stratified by patient age (below or greater than or equal to the median age of 57 years) and surgical status (patient with surgery during hospitalization or medical patient only). Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Disclosures

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Edelson has received research support from the National Heart, Lung, and Blood Institute (K23 HL097157), Philips (Andover, MA), the American Heart Association (Dallas, TX), Laerdal Medical (Stavanger, Norway), and Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no conflicts of interest.

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

Study Setting and Protocol

We conducted an observational cohort study of consecutive adult general‐ward admissions from July 2011 through January 2013 at a 500‐bed, urban US teaching hospital. During the study period, no early‐warning scoring systems were in place on the hospital wards. Rapid response teams responding to altered mental status would do so without specific thresholds for activation. During this period, nurses on the general floors were expected to record each patient's GCS and RASS score in the electronic health record (EPIC Systems Corp., Verona, WI) as part of the routine patient assessment at least once every 12‐hour shift. AVPU assessments were extracted from the eye component of the GCS. The letter A was assigned to a GCS Eye score of 4 (opens eyes spontaneously), V to a score of 3 (opens eyes in response to voice), P to a score of 2 (opens eyes in response to painful stimuli), and U to a score of 1 (does not open eyes). To avoid comparison of mental‐status scores at different time points, only concurrent GCS and RASS scores, documented within 10 minutes of one another, were included in the analysis.

Location and time‐stamped GCS and RASS scores, demographics, and in‐hospital mortality data were obtained from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol and data‐collection mechanisms were approved by The University of Chicago Institutional Review Board (#16995A).

Statistical Analysis

Baseline admission characteristics were described using proportions (%) and measures of central tendency (mean, standard deviations [SD]; median, interquartile ranges [IQR]). Patient severity of illness at first ward observation was calculated using the MEWS.[19] All mental‐status observations during a patient's ward stay were included in the analysis. Odds ratios for 24‐hour mortality following an abnormal mental‐status score were calculated using generalized estimating equations, with an exchangeable correlation structure to account for the correlation of scores within the same patient, as more than 1 abnormal mental‐status score may have been documented within the 24 hours preceding death. Spearman's rank correlation coefficients () were used to estimate the correlation among AVPU, GCS, and RASS scores.

The predictive accuracies of AVPU, GCS, RASS, and the subscales of GCS were compared using the area under the receiver operating characteristic curve (AUC), with mortality within 24 hours of a mental‐status observation as the primary outcome and the mental‐status score as the predictor variable. Although AUCs are typically used as a measure of discriminative ability, this study used AUCs to summarize both sensitivity and specificity across a range of cutoffs, providing an overall measure of predictive accuracies across mental‐status scales. To estimate AUCs, the AVPU, GCS, and GCS subscales were entered into a logistic regression model as ordinal variables, whereas RASS was entered as a nominal variable due to its positive and negative components, and predicted probabilities were calculated. In addition, a combined model was fit where GCS and RASS were classified as categorical independent variables. AUCs were then calculated by utilizing the predicted probabilities from each logistic regression model using the trapezoidal rule.[20] A sensitivity analysis was performed to estimate the internal validity of the RASS model using 10‐fold cross‐validation.

Predefined subgroup analyses were performed that compared the accuracies of AVPU, GCS, and RASS for predicting 24‐hour mortality in patients above and below the median age of the study population, and between patients who underwent surgery during their admission or not (surgical vs medical). All tests of significance used a 2‐sided P value <0.05. All data analysis was performed using Stata version 13.0 (StataCorp, College Station, TX).

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

Baseline Characteristics of Hospital Admissions
  • NOTE: Characteristics are stratified at the hospital admission level. Abbreviations: IQR, interquartile range; MEWS, Modified Early Warning Score; n, number of observations; SD, standard deviation.

Total no. of admissions26,873
Total no. of unique patients17,660
Age, y, mean (SD)57 (17)
Female sex, n (%)14,293 (53)
Race, n (%) 
White10,516 (39)
Black12,580 (47)
Other/unknown3,777 (14)
Admission MEWS, median (IQR)1 (11)
Days on ward, median (IQR)5 (310)
Observations per person, per day, median (IQR)3 (24)
Underwent surgery during hospitalization, n (%)6,141 (23)
Deaths, n (%)417 (1.6)

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1
Score correlations between (1a) AVPU and GCS total, and between (1b) AVPU and RASS. Boxes indicate interquartile range (25th to 75th percentiles), whiskers indicate 5th to 95th percentiles, and diamonds indicate median. Each correlation is significant at P < 0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Odds of Mortality Within 24 Hours of an Abnormal Mental‐Status Score
Mental‐status ScoreObservations, n (%)Odds Ratio for Mortality (95% CI)
  • NOTE: Odds ratios, with 95% CIs, comparing the probability of mortality within 24 hours of an abnormal mental‐status score to the probability of mortality within 24 hours of a normal mental‐status score (Reference). All calculations control for clustering of observations within the same admission. All odds ratios were significant at P<0.001. Abbreviations: AVPU, Alert‐Voice‐Pain‐Unresponsive; CI, confidence interval; GCS, Glascow Coma Scale; n, number of observations; RASS, Richmond Agitation Sedation Scale.

GCS Eye (AVPU)  
4 (alert)289,857 (98)Reference
<4 (not alert)6,117 (2)33.8 (23.947.9)
GCS Verbal  
5277,862 (94)Reference
411,258 (4)4.7 (2.87.9)
<46,854 (2)52.7 (38.073.2)
GCS Motor  
6287,441 (97)Reference
<68,533 (3)41.8 (30.756.9)
GCS total  
15276,042 (93)Reference
13, 1412,437 (4)5.2 (3.38.3)
<137,495 (3)55.5 (40.077.1)
RASS  
>06,867 (2)8.5 (5.613.0)
0275,708 (93)Reference
<013,339 (5)25.8 (19.234.6)

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2
Predictive accuracies of mental‐status scales (and GCS subscales) for mortality within 24 hours of a mental‐status observation (*P < 0.001). AUC with whiskers indicating 95% confidence intervals for predicting mortality occurring within 24 hours of a mental‐status observation. AUCs are shown for each mental‐status scale, for the combination of GCS and RASS, and for the 3 subscales of the GCS. Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.
Figure 3
Predictive accuracies of AVPU, GCS, and RASS for mortality within 24 hours of a mental‐status observation. Subgroup analysis is based on age and surgical status (*P < 0.05, **P < 0.001). AUC with whiskers indicating 95% CI for predicting mortality occurring within 24 hours of a mental‐status observation, analyzed at the observation level, and stratified by patient age (below or greater than or equal to the median age of 57 years) and surgical status (patient with surgery during hospitalization or medical patient only). Abbreviations: 95% CI, 95% confidence interval; AUC, area under the receiver operating characteristic curve; AVPU, Alert‐Voice‐Pain‐Unresponsive; GCS, Glascow Coma Scale; RASS, Richmond Agitation Sedation Scale.

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Disclosures

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Edelson has received research support from the National Heart, Lung, and Blood Institute (K23 HL097157), Philips (Andover, MA), the American Heart Association (Dallas, TX), Laerdal Medical (Stavanger, Norway), and Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no conflicts of interest.

References
  1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):17531762.
  2. Pompei P, Foreman M, Rudberg MA, Inouye SK, Braund V, Cassel CK. Delirium in hospitalized older persons: outcomes and predictors. J Am Geriatr Soc. 1994;42(8):809815.
  3. Siddiqi N, House AO, Holmes JD. Occurrence and outcome of delirium in medical in‐patients: a systematic literature review. Age Ageing. 2006;35(4):350364.
  4. Levkoff SE, Evans DA, Liptzin B, et al. Delirium. The occurrence and persistence of symptoms among elderly hospitalized patients. Arch Intern Med. 1992;152(2):334340.
  5. Dyer CB, Ashton CM, Teasdale TA. Postoperative delirium. A review of 80 primary data‐collection studies. Arch Intern Med. 1995;155(5):461465.
  6. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM Nurses' recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):24672473.
  7. Armstrong SC, Cozza KL, Watanabe KS. The misdiagnosis of delirium. Psychosomatics. 1997;38(5):433439.
  8. Ely EW, Stephens RK, Jackson JC, et al. Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professionals. Crit Care Med. 2004;32(1):106112.
  9. Farrell KR, Ganzini L. Misdiagnosing delirium as depression in medically ill elderly patients. Arch Intern Med. 1995;155(22):24592464.
  10. Simpson CJ. Doctors and nurses use of the word confused. Br J Psychiatry. 1984;145:441443.
  11. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation‐Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):13381344.
  12. Teasdale G, Jennett B. Assessment and prognosis of coma after head injury. Acta Neurochir (Wien). 1976;34(1–4):4555.
  13. Bastos PG, Sun X, Wagner DP, Wu AW, Knaus WA. Glasgow Coma Scale score in the evaluation of outcome in the intensive care unit: findings from the Acute Physiology and Chronic Health Evaluation III study. Crit Care Med. 1993;21(10):14591465.
  14. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
  15. Menegazzi JJ, Davis EA, Sucov AN, Paris PM. Reliability of the Glasgow Coma Scale when used by emergency physicians and paramedics. J Trauma. 1993;34(1):4648.
  16. Alexander RH, Proctor HJ; American College of Surgeons. Committee on Trauma. Advanced Trauma Life Support Program For Physicians: ATLS. 5th ed. Chicago, IL: American College of Surgeons; 1993.
  17. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation. 2008;77(2):170179.
  18. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, Higgins B. A review, and performance evaluation, of single‐parameter “track and trigger” systems. Resuscitation. 2008;79(1):1121.
  19. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning score in medical admissions. QJM. 2001;94(10):521526.
  20. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  21. McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glascow Coma Scale. Anaesthesia. 2004;59(1):3437.
  22. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  23. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  24. Peterson JF, Pun BT, Dittus RS, et al. Delirium and its motoric subtypes: a study of 614 critically ill patients. J Am Geriatr Soc. 2006;54(3):479484.
  25. Angles EM, Robinson TN, Biffl WL, et al. Risk factors for delirium after major trauma. Am J Surg. 2008;196(6):864869.
  26. Meagher DJ, O'Hanlon D, O'Mahony E, Casey PR, Trzepacz PT. Relationship between symptoms and motoric subtype of delirium. J Neuropsychiatry Clin Neurosci. 2000;12(1):5156.
  27. Yang FM, Marcantonio ER, Inouye SK, et al. Phenomenological subtypes of delirium in older persons: patterns, prevalence, and prognosis. Psychosomatics. 2009;50(3):248254.
  28. Kiely DK, Jones RN, Bergmann MA, Marcantonio ER. Association between psychomotor activity delirium subtypes and mortality among newly admitted post‐acute facility patients. J Gerontol A Biol Sci Med Sci. 2007;62(2):174179.
  29. Marcantonio E, Ta T, Duthie E, Resnick NM. Delirium severity and psychomotor types: their relationship with outcomes after hip fracture repair. J Am Geriatr Soc. 2002;50(5):850857.
  30. Rowley G, Fielding K. Reliability and accuracy of the Glasgow Coma Scale with experienced and inexperienced users. Lancet. 1991;337(8740):535538.
  31. Pun BT, Gordon SM, Peterson JF, et al. Large‐scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med. 2005;33(6):11991205.
  32. Vasilevskis EE, Morandi A, Boehm L, et al. Delirium and sedation recognition using validated instruments: reliability of bedside intensive care unit nursing assessments from 2007 to 2010. J Am Geriatr Soc. 2011;59(suppl 2):S249S255.
References
  1. Ely EW, Shintani A, Truman B, et al. Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA. 2004;291(14):17531762.
  2. Pompei P, Foreman M, Rudberg MA, Inouye SK, Braund V, Cassel CK. Delirium in hospitalized older persons: outcomes and predictors. J Am Geriatr Soc. 1994;42(8):809815.
  3. Siddiqi N, House AO, Holmes JD. Occurrence and outcome of delirium in medical in‐patients: a systematic literature review. Age Ageing. 2006;35(4):350364.
  4. Levkoff SE, Evans DA, Liptzin B, et al. Delirium. The occurrence and persistence of symptoms among elderly hospitalized patients. Arch Intern Med. 1992;152(2):334340.
  5. Dyer CB, Ashton CM, Teasdale TA. Postoperative delirium. A review of 80 primary data‐collection studies. Arch Intern Med. 1995;155(5):461465.
  6. Inouye SK, Foreman MD, Mion LC, Katz KH, Cooney LM Nurses' recognition of delirium and its symptoms: comparison of nurse and researcher ratings. Arch Intern Med. 2001;161(20):24672473.
  7. Armstrong SC, Cozza KL, Watanabe KS. The misdiagnosis of delirium. Psychosomatics. 1997;38(5):433439.
  8. Ely EW, Stephens RK, Jackson JC, et al. Current opinions regarding the importance, diagnosis, and management of delirium in the intensive care unit: a survey of 912 healthcare professionals. Crit Care Med. 2004;32(1):106112.
  9. Farrell KR, Ganzini L. Misdiagnosing delirium as depression in medically ill elderly patients. Arch Intern Med. 1995;155(22):24592464.
  10. Simpson CJ. Doctors and nurses use of the word confused. Br J Psychiatry. 1984;145:441443.
  11. Sessler CN, Gosnell MS, Grap MJ, et al. The Richmond Agitation‐Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med. 2002;166(10):13381344.
  12. Teasdale G, Jennett B. Assessment and prognosis of coma after head injury. Acta Neurochir (Wien). 1976;34(1–4):4555.
  13. Bastos PG, Sun X, Wagner DP, Wu AW, Knaus WA. Glasgow Coma Scale score in the evaluation of outcome in the intensive care unit: findings from the Acute Physiology and Chronic Health Evaluation III study. Crit Care Med. 1993;21(10):14591465.
  14. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
  15. Menegazzi JJ, Davis EA, Sucov AN, Paris PM. Reliability of the Glasgow Coma Scale when used by emergency physicians and paramedics. J Trauma. 1993;34(1):4648.
  16. Alexander RH, Proctor HJ; American College of Surgeons. Committee on Trauma. Advanced Trauma Life Support Program For Physicians: ATLS. 5th ed. Chicago, IL: American College of Surgeons; 1993.
  17. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI. Review and performance evaluation of aggregate weighted 'track and trigger' systems. Resuscitation. 2008;77(2):170179.
  18. Smith GB, Prytherch DR, Schmidt PE, Featherstone PI, Higgins B. A review, and performance evaluation, of single‐parameter “track and trigger” systems. Resuscitation. 2008;79(1):1121.
  19. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning score in medical admissions. QJM. 2001;94(10):521526.
  20. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  21. McNarry AF, Goldhill DR. Simple bedside assessment of level of consciousness: comparison of two simple assessment scales with the Glascow Coma Scale. Anaesthesia. 2004;59(1):3437.
  22. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  23. Churpek MM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5):564568.
  24. Peterson JF, Pun BT, Dittus RS, et al. Delirium and its motoric subtypes: a study of 614 critically ill patients. J Am Geriatr Soc. 2006;54(3):479484.
  25. Angles EM, Robinson TN, Biffl WL, et al. Risk factors for delirium after major trauma. Am J Surg. 2008;196(6):864869.
  26. Meagher DJ, O'Hanlon D, O'Mahony E, Casey PR, Trzepacz PT. Relationship between symptoms and motoric subtype of delirium. J Neuropsychiatry Clin Neurosci. 2000;12(1):5156.
  27. Yang FM, Marcantonio ER, Inouye SK, et al. Phenomenological subtypes of delirium in older persons: patterns, prevalence, and prognosis. Psychosomatics. 2009;50(3):248254.
  28. Kiely DK, Jones RN, Bergmann MA, Marcantonio ER. Association between psychomotor activity delirium subtypes and mortality among newly admitted post‐acute facility patients. J Gerontol A Biol Sci Med Sci. 2007;62(2):174179.
  29. Marcantonio E, Ta T, Duthie E, Resnick NM. Delirium severity and psychomotor types: their relationship with outcomes after hip fracture repair. J Am Geriatr Soc. 2002;50(5):850857.
  30. Rowley G, Fielding K. Reliability and accuracy of the Glasgow Coma Scale with experienced and inexperienced users. Lancet. 1991;337(8740):535538.
  31. Pun BT, Gordon SM, Peterson JF, et al. Large‐scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med. 2005;33(6):11991205.
  32. Vasilevskis EE, Morandi A, Boehm L, et al. Delirium and sedation recognition using validated instruments: reliability of bedside intensive care unit nursing assessments from 2007 to 2010. J Am Geriatr Soc. 2011;59(suppl 2):S249S255.
Issue
Journal of Hospital Medicine - 10(10)
Issue
Journal of Hospital Medicine - 10(10)
Page Number
658-663
Page Number
658-663
Publications
Publications
Article Type
Display Headline
Comparison of mental‐status scales for predicting mortality on the general wards
Display Headline
Comparison of mental‐status scales for predicting mortality on the general wards
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, Section of Hospital Medicine, University of Chicago Medical Center, 5841 S Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773‐834‐2191; Fax: 773‐795‐7398; E‐mail: dperes@uchicago.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

OSA and Outcomes in Ward Patients

Article Type
Changed
Tue, 05/16/2017 - 23:06
Display Headline
Obstructive sleep apnea and adverse outcomes in surgical and nonsurgical patients on the wards

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

Setting and Study Population

This observational cohort study was performed at an academic tertiary care medical center with approximately 500 beds. Data were obtained from all adult patients hospitalized on the wards between November 1, 2008 and October 1, 2013. Our hospital has utilized an RRT, led by a critical care nurse and respiratory therapist with hospitalist and pharmacist consultation available upon request, since 2008. This team is separate from the team that responds to a cardiac arrest. Criteria for RRT activation include tachypnea, tachycardia, hypotension, and staff worry, but specific vital sign thresholds are not specified.

The study analyzed deidentified data from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol was approved by the University of Chicago Institutional Review Board (IRB #16995A).

Data Collection

Patient age, sex, race, body mass index (BMI), and location prior to ward admission (ie, whether they were admitted from the emergency department, transferred from the ICU, or directly admitted from clinic or home) were collected. Patients who underwent surgery during their admission were identified using the hospital's admission‐transfer‐discharge database. In addition, routinely collected vital signs (eg, respiratory rate, blood pressure, heart rate) were obtained from the electronic health record (Epic, Verona, WI). To determine severity of illness, the first set of vital signs measured on hospital presentation were utilized to calculate the cardiac arrest risk triage (CART) score, a vital‐signbased early warning score we previously developed and validated for predicting adverse events in our population.[17] The CART score ranges from 0 to 57, with points assigned for abnormalities in respiratory rate, heart rate, diastolic blood pressure, and age. If any vital sign was missing, the next available measurement was pulled into the set. If any vital sign remained missing after this change, the median value for that particular location (ie, wards, ICU, or emergency department) was imputed as previously described.[18, 19]

Patients with OSA were identified by the following International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes using inpatient and outpatient medical records: 278.03, 327.20, 327.23, 327.29, 780.51, 780.53, and 780.57 (Table 1). Data on other patient comorbidities, including coronary artery disease, congestive heart failure, arrhythmias, uncomplicated and complicated diabetes mellitus, hypertension, and cerebrovascular disease were collected using specific ICD‐9‐CM codes from both inpatient and outpatient records. Information on insurance payer was also collected from the hospital's billing database. Insurance payers were grouped into the following categories: private payer, Medicare/Medicaid, and no insurance. Patients with both public and private payers were counted as being privately insured.

Diagnosis Codes and Prevalence of Obstructive Sleep Apnea
Diagnosis CodeDescription% of Sleep Apnea Diagnosesa
  • Percentages add to >100% as a small number of patients carried more than 1 sleep apnea diagnosis.

327.23Obstructive sleep apnea65.6
780.57Unspecified sleep apnea19.4
780.53Hypersomnia with sleep apnea, unspecified11.7
780.51Insomnia with sleep apnea, unspecified1.5
327.2Organic sleep apnea, unspecified0.2
278.03Obesity hypoventilation syndrome1.7

Outcomes

The primary outcome of the study was in‐hospital mortality. Secondary outcomes included length of stay, RRT activation, transfer to the ICU, endotracheal intubation, cardiac arrest (defined as a loss of pulse with attempted resuscitation) on the wards, and a composite outcome of RRT activation, ICU transfer, and death. Because cardiac arrests on the wards result either in death or ICU transfer following successful resuscitation, this variable was omitted from the composite outcome. Cardiac arrests were identified using a prospectively validated quality improvement database that has been described previously.[20] ICU transfer was identified using the hospital's admission‐transfer‐discharge database. Only the index cardiac arrest, intubation, RRT, or ICU transfer for each admission was used in the study, but more than 1 type of outcome could occur for each patient (eg, a patient who died following an unsuccessful resuscitation attempt would count as both a cardiac arrest and a death).

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and 2 statistics, as appropriate. Unadjusted logistic regression models were fit to estimate the change in odds of each adverse event and a composite outcome of any event for patient admissions with OSA compared to those without OSA. Adjusted logistic regression models were then fit for each outcome to control for patient characteristics (age, sex, BMI, insurance status, and individual comorbidities), location immediately prior to ward admission, and admission severity of illness (as measured by CART score). In the adjusted model, CART score, age, and number of comorbidities were entered linearly, with the addition of squared terms for age and CART score, as these variables showed nonlinear associations with the outcomes of interest. Race, surgical status, insurance payer, location prior to ward, and BMI (underweight, <18.5 kg/m2; normal weight, 18.524.9 kg/m2; overweight, 25.029.9 kg/m2; obese, 3039.9 kg/m2; and severely obese, (40 kg/m2) were modeled as categorical variables.

Given that an individual patient could experience multiple hospitalizations during the study period, we performed a sensitivity analysis of all adjusted and unadjusted models using a single randomly selected hospitalization for each unique patient. In addition, we performed a sensitivity analysis of all patients who were not admitted to the ICU prior to their ward stay. Finally, we performed subgroup analyses of all unadjusted and adjusted models for each BMI category and surgical status.

All tests of significance used a 2‐sided P value <0.05. Statistical analyses were completed using Stata version 12.0 (StataCorp, College Station, TX).

RESULTS

Patient Characteristics

During the study period, 93,676 patient admissions from 53,150 unique patients resulted in the occurrence of 1,069 RRT activations, 6,305 ICU transfers, and 1,239 in‐hospital deaths. Within our sample, 40,034 patients had at least 1 inpatient record and at least 1 outpatient record. OSA diagnosis was present in 5,625 patients (10.6% of the total sample), with 4,748 patients having an OSA diagnosis code entered during a hospitalization, 2,143 with an OSA diagnosis code entered during an outpatient encounter, and 877 with both inpatient and outpatient diagnosis codes. These patients identified as having OSA contributed 12,745 (13.6%) hospital admissions.

Patients with an OSA diagnosis were more likely to be older (median age 59 years [interquartile range 4968] vs 55 years [3868]), male (49% vs 42%), overweight or obese (88% vs 62%), and more likely to carry diagnoses of diabetes (53.8% vs 25.5%), hypertension (45.3% vs 18.2%), arrhythmias (44.4% vs 26.7%), coronary artery disease (46.8% vs 23.5%), heart failure (35.8% vs 13.5%), and cerebrovascular disease (13.5% vs 8.1%) than patients without an OSA diagnosis (all comparisons significant, P < 0.001) (Table 2).

Patient Characteristics for Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; IQR, interquartile range; OSA, obstructive sleep apnea.

Age, y, median (IQR)59 (4968)55 (3868)<0.001
Female, n (%)6,514 (51%)47,202 (58%)<0.001
Race, n (%)  <0.001
White4,205 (33%)30,119 (37%) 
Black/African American7,024 (55%)38,561 (48%) 
Asian561 (4.4%)3,419 (4.2%) 
American Indian or Native Alaskan20 (0.2%)113 (0.1%) 
More than 1 race127 (1%)843 (1%) 
Race unknown808 (6%)7,876 (10%) 
Insurance status, n (%)  <0.001
Private4,484 (35%)32,467 (40%) 
Medicare/Medicaid8,201 (64%)42,208 (58%) 
Uninsured53 (0.4%)1,190 (1%) 
Unknown4 (<0.1%)16 (<0.1%) 
Location prior to wards, n (%)  <0.001
ICU1,400 (11%)8,065 (10%) 
Emergency department4,633 (36%)25,170 (31%) 
Ambulatory admission6,712 (53%)47,696 (59%) 
Body mass index, kg/m2, n (%)  <0.001
Normal (18.525)1,431 (11%)26,560 (33%) 
Underweight (<18.5)122 (1%)4,256 (5%) 
Overweight (2530)2,484 (20%)23,761 (29%) 
Obese (3040)4,959 (39%)19,132 (24%) 
Severely obese (40)3,745 (29%)7,171 (9%) 
Initial cardiac arrest risk triage score, median (IQR)4 (09)4 (09)<0.001
Underwent surgery, n (%)4,482 (35%)28,843 (36%)0.3
Comorbidities   
Number of comorbidities, median (IQR)2 (14)1 (02)<0.001
Arrhythmia5,659 (44%)21,581 (27%)<0.001
Diabetes mellitus6,855 (54%)20,641 (26%)<0.001
Hypertension5,777 (45%)14,728 (18%)<0.001
Coronary artery disease5,958 (47%)18,979 (23%)<0.001
Cerebrovascular accident1,725 (14%)6,556 (8%)<0.001
Congestive heart failure4,559 (36%)10,919 (13%)<0.001

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea.

  • Experiencing rapid response team call, ICU transfer, or in‐hospital death.

Outcomes, n (%)   
Composite outcomea1,137 (9%)5,792 (7%)<0.001
In‐hospital death144 (1.1%)1,095 (1.4%)0.04
Rapid response team call188 (1.5%)881 (1.1%)<0.001
ICU transfer1,045 (8%)5,260 (7%)<0.001
Cardiac arrest413 (0.5%)73 (0.6%)0.36
Figure 1
Adjusted models for the association of OSA with clinical deterioration outcomes. Odds of RRT activation, intubation, ICU transfer, cardiac arrest, and in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis. Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea; RRT, rapid response team.

Sensitivity Analyses

The sensitivity analysis involving 1 randomly selected hospitalization per patient included a total of 53,150 patients. The results were similar to the main analysis, with adjusted odds of 1.01 (0.77‐1.32) for RRT activation, 0.86 (0.76‐0.96) for ICU transfer, and 0.69 (0.53‐0.89) for inpatient mortality. An additional sensitivity analysis included only patients who were not admitted to the ICU prior to their ward stay. This analysis included 84,211 hospitalizations and demonstrated similar findings, with adjusted odds of 0.70 for in‐hospital mortality (0.57‐0.87). Adjusted odds for RRT activation (OR: 1.12 [0.92‐1.37]) and ICU transfer (OR: 0.88 [0.81‐0.96] were also similar to the results of our main analysis.

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2
Adjusted models for the association of OSA with death, by surgical status and BMI. Odds of in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis, stratified by surgical status and BMI. Abbreviations: BMI, body mass index; OSA, obstructive sleep apnea.

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m2 (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.

Acknowledgements

The authors thank Nicole Babuskow for administrative support, as well as Brian Furner and Timothy Holper for assistance with data acquisition.

Disclosures: Study concept and design: P.L., D.P.E, B.M., M.C.; acquisition of data: P.L.; analysis and interpretation of data: all authors; first drafting of the manuscript: P.L.; critical revision of the manuscript for important intellectual content: all authors; statistical analysis: P.L., F.Z., M.C.; obtained funding: D.P.E., M.C.; administrative, technical, and material support: F.Z., D.P.E.; study supervision: D.P.E, B.M., M.C.; data access and responsibility: P.L. and M.C. had full access to all the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek and Dr. Edelson are both supported by career development awards from the National Heart, Lung, and Blood Institute (K08 HL121080 and K23 HL097157, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics.

Files
References
  1. Peppard PE, Young T, Barnet JH, Palta M, Hagen EW, Hla KM. Increased prevalence of sleep‐disordered breathing in adults. Am J Epidemiol. 2013;177(9):10061014.
  2. Marin JM, Carrizo SJ, Vicente E, Agusti AGN. Long‐term cardiovascular outcomes in men with obstructive sleep apnoea‐hypopnoea with or without treatment with continuous positive airway pressure: an observational study. Lancet. 2005;365(9464):10461053.
  3. Peppard PE, Young T, Palta M, Skatrud J. Prospective study of the association between sleep‐disordered breathing and hypertension. N Engl J Med. 2000;342(19):13781384.
  4. Yaggi HK, Concato J, Kernan WN, Lichtman JH, Brass LM, Mohsenin V. Obstructive sleep apnea as a risk factor for stroke and death. N Engl J Med. 2005;353(19):20342041.
  5. Kendzerska T, Gershon AS, Hawker G, Leung RS, Tomlinson G. Obstructive sleep apnea and risk of cardiovascular events and all‐cause mortality: a decade‐long historical cohort study. PLoS Med. 2014;11(2):e1001599.
  6. Marshall NS, Wong KK, Liu PY, Cullen SRJ, Knuiman MW, Grunstein RR. Sleep apnea as an independent risk factor for all‐cause mortality: the Busselton Health Study. Sleep. 2008;31(8):10791085.
  7. Kaw R, Pasupuleti V, Walker E, Ramaswamy A, Foldvary‐Schafer N. Postoperative complications in patients with obstructive sleep apnea. Chest. 2012;141(2):436441.
  8. Kaw R, Chung F, Pasupuleti V, Mehta J, Gay PC, Hernandez A. Meta‐analysis of the association between obstructive sleep apnoea and postoperative outcome. Br J Anaesth. 2012;109(6):897906.
  9. Memtsoudis SG, Stundner O, Rasul R, et al. The impact of sleep apnea on postoperative utilization of resources and adverse outcomes. Anesth Analg. 2014;118(2):407418.
  10. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after bariatric surgery: analysis of the nationwide inpatient sample. Obes Surg. 2013;23(11):18421851.
  11. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after elective surgery: analysis of the nationwide inpatient sample. Chest. 2013;144:903914.
  12. Lindenauer PK, Stefan MS, Johnson KG, Priya A, Pekow PS, Rothberg MB. Prevalence, treatment and outcomes associated with obstructive sleep apnea among patients hospitalized with pneumonia. Chest. 2014;145(5):10321038.
  13. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PLoS One. 2013;8(1):e54807.
  14. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250254.
  15. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diazepam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):6569.
  16. Dolly FR, Block AJ. Effect of flurazepam on sleep‐disordered breathing and nocturnal oxygen desaturation in asymptomatic subjects. Am J Med. 1982;73(2):239243.
  17. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs*. Crit Care Med. 2012;40(7):21022108.
  18. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*. Crit Care Med. 2014;42(4):841848.
  19. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):16191636.
  20. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case‐control study. Chest. 2012;141(5):11701176.
  21. Memtsoudis SG, Bombardieri AM, Ma Y, Walz JM, Chiu YL, Mazumdar M. Mortality of patients with respiratory insufficiency and adult respiratory distress syndrome after surgery: the obesity paradox. J Intensive Care Med. 2012;27(4):306311.
  22. Bucholz EM, Rathore SS, Reid KJ, et al. Body mass index and mortality in acute myocardial infarction patients. Am J Med. 2012(8);125:796803.
  23. Murry CE, Jennings RB, Reimer KA. Preconditioning with ischemia: a delay of lethal cell injury in ischemic myocardium. Circulation. 1986;74(5):11241136.
  24. Murry CE, Richard VJ, Reimer KA, Jennings RB. Ischemic preconditioning slows energy metabolism and delays ultrastructural damage during a sustained ischemic episode. Circ Res. 1990;66(4):913931.
  25. Hu X, Yu SP, Fraser JL, et al. Transplantation of hypoxia‐preconditioned mesenchymal stem cells improves infarcted heart function via enhanced survival of implanted cells and angiogenesis. J Thorac Cardiovasc Surg. 2008;135(4):799808.
  26. Yu X, Lu C, Liu H, et al. Hypoxic preconditioning with cobalt of bone marrow mesenchymal stem cells improves cell migration and enhances therapy for treatment of ischemic acute kidney injury. PLoS One. 2013;8(5):e62703.
  27. Francis KR, Wei L. Human embryonic stem cell neural differentiation and enhanced cell survival promoted by hypoxic preconditioning. Cell Death Dis. 2010;1:e22.
  28. Kamota T, Li TS, Morikage N, et al. Ischemic pre‐conditioning enhances the mobilization and recruitment of bone marrow stem cells to protect against ischemia/reperfusion injury in the late phase. J Am Coll Cardiol. 2009;53(19):18141822.
  29. Hu X, Wei L, Taylor TM, et al. Hypoxic preconditioning enhances bone marrow mesenchymal stem cell migration via Kv2.1 channel and FAK activation. Am J Physiol Cell Physiol. 2011;301(2):C362C372.
  30. Theus MH, Wei L, Cui L, et al. In vitro hypoxic preconditioning of embryonic stem cells as a strategy of promoting cell survival and functional benefits after transplantation into the ischemic rat brain. Exp Neurol. 2008;210(2):656670.
  31. Wei L, Fraser JL, Lu ZY, Hu X, Yu SP. Transplantation of hypoxia preconditioned bone marrow mesenchymal stem cells enhances angiogenesis and neurogenesis after cerebral ischemia in rats. Neurobiol Dis. 2012;46(3):635645.
  32. Kharbanda RK, Nielsen TT, Redington AN. Translation of remote ischaemic preconditioning into clinical practice. Lancet. 2009;374(9700):15571565.
  33. Schmidt MR, Pryds K, Bøtker HE. Novel adjunctive treatments of myocardial infarction. World J Cardiol. 2014;6(6):434443.
  34. Ara J, Montpellier S. Hypoxic‐preconditioning enhances the regenerative capacity of neural stem/progenitors in subventricular zone of newborn piglet brain. Stem Cell Res. 2013;11(2):669686.
  35. Foster GP, Giri PC, Rogers DM, Larson SR, Anholm JD. Ischemic preconditioning improves oxygen saturation and attenuates hypoxic pulmonary vasoconstriction at high altitude. High Alt Med Biol. 2014;15(2):155161.
  36. Jean‐St‐Michel E, Manlhiot C, Li J, et al. Remote preconditioning improves maximal performance in highly trained athletes. Med Sci Sports Exerc. 2011;43(7):12801286.
  37. Durán J, Esnaola S, Rubio R, Iztueta Á. Obstructive sleep apnea‐hypopnea and related clinical features in a population‐based sample of subjects aged 30 to 70 yr. Am J Respir Crit Care Med. 2001;163(3 pt 1):685689.
  38. Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep‐disordered breathing among middle‐aged adults. N Engl J Med. 1993;328(17):12301235.
  39. Young T, Peppard PE, Gottlieb DJ. Epidemiology of obstructive sleep apnea: a population health perspective. Am J Respir Crit Care Med. 2002;165(9):12171239.
  40. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
Article PDF
Issue
Journal of Hospital Medicine - 10(9)
Publications
Page Number
592-598
Sections
Files
Files
Article PDF
Article PDF

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

Setting and Study Population

This observational cohort study was performed at an academic tertiary care medical center with approximately 500 beds. Data were obtained from all adult patients hospitalized on the wards between November 1, 2008 and October 1, 2013. Our hospital has utilized an RRT, led by a critical care nurse and respiratory therapist with hospitalist and pharmacist consultation available upon request, since 2008. This team is separate from the team that responds to a cardiac arrest. Criteria for RRT activation include tachypnea, tachycardia, hypotension, and staff worry, but specific vital sign thresholds are not specified.

The study analyzed deidentified data from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol was approved by the University of Chicago Institutional Review Board (IRB #16995A).

Data Collection

Patient age, sex, race, body mass index (BMI), and location prior to ward admission (ie, whether they were admitted from the emergency department, transferred from the ICU, or directly admitted from clinic or home) were collected. Patients who underwent surgery during their admission were identified using the hospital's admission‐transfer‐discharge database. In addition, routinely collected vital signs (eg, respiratory rate, blood pressure, heart rate) were obtained from the electronic health record (Epic, Verona, WI). To determine severity of illness, the first set of vital signs measured on hospital presentation were utilized to calculate the cardiac arrest risk triage (CART) score, a vital‐signbased early warning score we previously developed and validated for predicting adverse events in our population.[17] The CART score ranges from 0 to 57, with points assigned for abnormalities in respiratory rate, heart rate, diastolic blood pressure, and age. If any vital sign was missing, the next available measurement was pulled into the set. If any vital sign remained missing after this change, the median value for that particular location (ie, wards, ICU, or emergency department) was imputed as previously described.[18, 19]

Patients with OSA were identified by the following International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes using inpatient and outpatient medical records: 278.03, 327.20, 327.23, 327.29, 780.51, 780.53, and 780.57 (Table 1). Data on other patient comorbidities, including coronary artery disease, congestive heart failure, arrhythmias, uncomplicated and complicated diabetes mellitus, hypertension, and cerebrovascular disease were collected using specific ICD‐9‐CM codes from both inpatient and outpatient records. Information on insurance payer was also collected from the hospital's billing database. Insurance payers were grouped into the following categories: private payer, Medicare/Medicaid, and no insurance. Patients with both public and private payers were counted as being privately insured.

Diagnosis Codes and Prevalence of Obstructive Sleep Apnea
Diagnosis CodeDescription% of Sleep Apnea Diagnosesa
  • Percentages add to >100% as a small number of patients carried more than 1 sleep apnea diagnosis.

327.23Obstructive sleep apnea65.6
780.57Unspecified sleep apnea19.4
780.53Hypersomnia with sleep apnea, unspecified11.7
780.51Insomnia with sleep apnea, unspecified1.5
327.2Organic sleep apnea, unspecified0.2
278.03Obesity hypoventilation syndrome1.7

Outcomes

The primary outcome of the study was in‐hospital mortality. Secondary outcomes included length of stay, RRT activation, transfer to the ICU, endotracheal intubation, cardiac arrest (defined as a loss of pulse with attempted resuscitation) on the wards, and a composite outcome of RRT activation, ICU transfer, and death. Because cardiac arrests on the wards result either in death or ICU transfer following successful resuscitation, this variable was omitted from the composite outcome. Cardiac arrests were identified using a prospectively validated quality improvement database that has been described previously.[20] ICU transfer was identified using the hospital's admission‐transfer‐discharge database. Only the index cardiac arrest, intubation, RRT, or ICU transfer for each admission was used in the study, but more than 1 type of outcome could occur for each patient (eg, a patient who died following an unsuccessful resuscitation attempt would count as both a cardiac arrest and a death).

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and 2 statistics, as appropriate. Unadjusted logistic regression models were fit to estimate the change in odds of each adverse event and a composite outcome of any event for patient admissions with OSA compared to those without OSA. Adjusted logistic regression models were then fit for each outcome to control for patient characteristics (age, sex, BMI, insurance status, and individual comorbidities), location immediately prior to ward admission, and admission severity of illness (as measured by CART score). In the adjusted model, CART score, age, and number of comorbidities were entered linearly, with the addition of squared terms for age and CART score, as these variables showed nonlinear associations with the outcomes of interest. Race, surgical status, insurance payer, location prior to ward, and BMI (underweight, <18.5 kg/m2; normal weight, 18.524.9 kg/m2; overweight, 25.029.9 kg/m2; obese, 3039.9 kg/m2; and severely obese, (40 kg/m2) were modeled as categorical variables.

Given that an individual patient could experience multiple hospitalizations during the study period, we performed a sensitivity analysis of all adjusted and unadjusted models using a single randomly selected hospitalization for each unique patient. In addition, we performed a sensitivity analysis of all patients who were not admitted to the ICU prior to their ward stay. Finally, we performed subgroup analyses of all unadjusted and adjusted models for each BMI category and surgical status.

All tests of significance used a 2‐sided P value <0.05. Statistical analyses were completed using Stata version 12.0 (StataCorp, College Station, TX).

RESULTS

Patient Characteristics

During the study period, 93,676 patient admissions from 53,150 unique patients resulted in the occurrence of 1,069 RRT activations, 6,305 ICU transfers, and 1,239 in‐hospital deaths. Within our sample, 40,034 patients had at least 1 inpatient record and at least 1 outpatient record. OSA diagnosis was present in 5,625 patients (10.6% of the total sample), with 4,748 patients having an OSA diagnosis code entered during a hospitalization, 2,143 with an OSA diagnosis code entered during an outpatient encounter, and 877 with both inpatient and outpatient diagnosis codes. These patients identified as having OSA contributed 12,745 (13.6%) hospital admissions.

Patients with an OSA diagnosis were more likely to be older (median age 59 years [interquartile range 4968] vs 55 years [3868]), male (49% vs 42%), overweight or obese (88% vs 62%), and more likely to carry diagnoses of diabetes (53.8% vs 25.5%), hypertension (45.3% vs 18.2%), arrhythmias (44.4% vs 26.7%), coronary artery disease (46.8% vs 23.5%), heart failure (35.8% vs 13.5%), and cerebrovascular disease (13.5% vs 8.1%) than patients without an OSA diagnosis (all comparisons significant, P < 0.001) (Table 2).

Patient Characteristics for Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; IQR, interquartile range; OSA, obstructive sleep apnea.

Age, y, median (IQR)59 (4968)55 (3868)<0.001
Female, n (%)6,514 (51%)47,202 (58%)<0.001
Race, n (%)  <0.001
White4,205 (33%)30,119 (37%) 
Black/African American7,024 (55%)38,561 (48%) 
Asian561 (4.4%)3,419 (4.2%) 
American Indian or Native Alaskan20 (0.2%)113 (0.1%) 
More than 1 race127 (1%)843 (1%) 
Race unknown808 (6%)7,876 (10%) 
Insurance status, n (%)  <0.001
Private4,484 (35%)32,467 (40%) 
Medicare/Medicaid8,201 (64%)42,208 (58%) 
Uninsured53 (0.4%)1,190 (1%) 
Unknown4 (<0.1%)16 (<0.1%) 
Location prior to wards, n (%)  <0.001
ICU1,400 (11%)8,065 (10%) 
Emergency department4,633 (36%)25,170 (31%) 
Ambulatory admission6,712 (53%)47,696 (59%) 
Body mass index, kg/m2, n (%)  <0.001
Normal (18.525)1,431 (11%)26,560 (33%) 
Underweight (<18.5)122 (1%)4,256 (5%) 
Overweight (2530)2,484 (20%)23,761 (29%) 
Obese (3040)4,959 (39%)19,132 (24%) 
Severely obese (40)3,745 (29%)7,171 (9%) 
Initial cardiac arrest risk triage score, median (IQR)4 (09)4 (09)<0.001
Underwent surgery, n (%)4,482 (35%)28,843 (36%)0.3
Comorbidities   
Number of comorbidities, median (IQR)2 (14)1 (02)<0.001
Arrhythmia5,659 (44%)21,581 (27%)<0.001
Diabetes mellitus6,855 (54%)20,641 (26%)<0.001
Hypertension5,777 (45%)14,728 (18%)<0.001
Coronary artery disease5,958 (47%)18,979 (23%)<0.001
Cerebrovascular accident1,725 (14%)6,556 (8%)<0.001
Congestive heart failure4,559 (36%)10,919 (13%)<0.001

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea.

  • Experiencing rapid response team call, ICU transfer, or in‐hospital death.

Outcomes, n (%)   
Composite outcomea1,137 (9%)5,792 (7%)<0.001
In‐hospital death144 (1.1%)1,095 (1.4%)0.04
Rapid response team call188 (1.5%)881 (1.1%)<0.001
ICU transfer1,045 (8%)5,260 (7%)<0.001
Cardiac arrest413 (0.5%)73 (0.6%)0.36
Figure 1
Adjusted models for the association of OSA with clinical deterioration outcomes. Odds of RRT activation, intubation, ICU transfer, cardiac arrest, and in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis. Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea; RRT, rapid response team.

Sensitivity Analyses

The sensitivity analysis involving 1 randomly selected hospitalization per patient included a total of 53,150 patients. The results were similar to the main analysis, with adjusted odds of 1.01 (0.77‐1.32) for RRT activation, 0.86 (0.76‐0.96) for ICU transfer, and 0.69 (0.53‐0.89) for inpatient mortality. An additional sensitivity analysis included only patients who were not admitted to the ICU prior to their ward stay. This analysis included 84,211 hospitalizations and demonstrated similar findings, with adjusted odds of 0.70 for in‐hospital mortality (0.57‐0.87). Adjusted odds for RRT activation (OR: 1.12 [0.92‐1.37]) and ICU transfer (OR: 0.88 [0.81‐0.96] were also similar to the results of our main analysis.

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2
Adjusted models for the association of OSA with death, by surgical status and BMI. Odds of in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis, stratified by surgical status and BMI. Abbreviations: BMI, body mass index; OSA, obstructive sleep apnea.

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m2 (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.

Acknowledgements

The authors thank Nicole Babuskow for administrative support, as well as Brian Furner and Timothy Holper for assistance with data acquisition.

Disclosures: Study concept and design: P.L., D.P.E, B.M., M.C.; acquisition of data: P.L.; analysis and interpretation of data: all authors; first drafting of the manuscript: P.L.; critical revision of the manuscript for important intellectual content: all authors; statistical analysis: P.L., F.Z., M.C.; obtained funding: D.P.E., M.C.; administrative, technical, and material support: F.Z., D.P.E.; study supervision: D.P.E, B.M., M.C.; data access and responsibility: P.L. and M.C. had full access to all the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek and Dr. Edelson are both supported by career development awards from the National Heart, Lung, and Blood Institute (K08 HL121080 and K23 HL097157, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics.

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

Setting and Study Population

This observational cohort study was performed at an academic tertiary care medical center with approximately 500 beds. Data were obtained from all adult patients hospitalized on the wards between November 1, 2008 and October 1, 2013. Our hospital has utilized an RRT, led by a critical care nurse and respiratory therapist with hospitalist and pharmacist consultation available upon request, since 2008. This team is separate from the team that responds to a cardiac arrest. Criteria for RRT activation include tachypnea, tachycardia, hypotension, and staff worry, but specific vital sign thresholds are not specified.

The study analyzed deidentified data from the hospital's Clinical Research Data Warehouse, which is maintained by the Center for Research Informatics at The University of Chicago. The study protocol was approved by the University of Chicago Institutional Review Board (IRB #16995A).

Data Collection

Patient age, sex, race, body mass index (BMI), and location prior to ward admission (ie, whether they were admitted from the emergency department, transferred from the ICU, or directly admitted from clinic or home) were collected. Patients who underwent surgery during their admission were identified using the hospital's admission‐transfer‐discharge database. In addition, routinely collected vital signs (eg, respiratory rate, blood pressure, heart rate) were obtained from the electronic health record (Epic, Verona, WI). To determine severity of illness, the first set of vital signs measured on hospital presentation were utilized to calculate the cardiac arrest risk triage (CART) score, a vital‐signbased early warning score we previously developed and validated for predicting adverse events in our population.[17] The CART score ranges from 0 to 57, with points assigned for abnormalities in respiratory rate, heart rate, diastolic blood pressure, and age. If any vital sign was missing, the next available measurement was pulled into the set. If any vital sign remained missing after this change, the median value for that particular location (ie, wards, ICU, or emergency department) was imputed as previously described.[18, 19]

Patients with OSA were identified by the following International Classification of Diseases, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes using inpatient and outpatient medical records: 278.03, 327.20, 327.23, 327.29, 780.51, 780.53, and 780.57 (Table 1). Data on other patient comorbidities, including coronary artery disease, congestive heart failure, arrhythmias, uncomplicated and complicated diabetes mellitus, hypertension, and cerebrovascular disease were collected using specific ICD‐9‐CM codes from both inpatient and outpatient records. Information on insurance payer was also collected from the hospital's billing database. Insurance payers were grouped into the following categories: private payer, Medicare/Medicaid, and no insurance. Patients with both public and private payers were counted as being privately insured.

Diagnosis Codes and Prevalence of Obstructive Sleep Apnea
Diagnosis CodeDescription% of Sleep Apnea Diagnosesa
  • Percentages add to >100% as a small number of patients carried more than 1 sleep apnea diagnosis.

327.23Obstructive sleep apnea65.6
780.57Unspecified sleep apnea19.4
780.53Hypersomnia with sleep apnea, unspecified11.7
780.51Insomnia with sleep apnea, unspecified1.5
327.2Organic sleep apnea, unspecified0.2
278.03Obesity hypoventilation syndrome1.7

Outcomes

The primary outcome of the study was in‐hospital mortality. Secondary outcomes included length of stay, RRT activation, transfer to the ICU, endotracheal intubation, cardiac arrest (defined as a loss of pulse with attempted resuscitation) on the wards, and a composite outcome of RRT activation, ICU transfer, and death. Because cardiac arrests on the wards result either in death or ICU transfer following successful resuscitation, this variable was omitted from the composite outcome. Cardiac arrests were identified using a prospectively validated quality improvement database that has been described previously.[20] ICU transfer was identified using the hospital's admission‐transfer‐discharge database. Only the index cardiac arrest, intubation, RRT, or ICU transfer for each admission was used in the study, but more than 1 type of outcome could occur for each patient (eg, a patient who died following an unsuccessful resuscitation attempt would count as both a cardiac arrest and a death).

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and 2 statistics, as appropriate. Unadjusted logistic regression models were fit to estimate the change in odds of each adverse event and a composite outcome of any event for patient admissions with OSA compared to those without OSA. Adjusted logistic regression models were then fit for each outcome to control for patient characteristics (age, sex, BMI, insurance status, and individual comorbidities), location immediately prior to ward admission, and admission severity of illness (as measured by CART score). In the adjusted model, CART score, age, and number of comorbidities were entered linearly, with the addition of squared terms for age and CART score, as these variables showed nonlinear associations with the outcomes of interest. Race, surgical status, insurance payer, location prior to ward, and BMI (underweight, <18.5 kg/m2; normal weight, 18.524.9 kg/m2; overweight, 25.029.9 kg/m2; obese, 3039.9 kg/m2; and severely obese, (40 kg/m2) were modeled as categorical variables.

Given that an individual patient could experience multiple hospitalizations during the study period, we performed a sensitivity analysis of all adjusted and unadjusted models using a single randomly selected hospitalization for each unique patient. In addition, we performed a sensitivity analysis of all patients who were not admitted to the ICU prior to their ward stay. Finally, we performed subgroup analyses of all unadjusted and adjusted models for each BMI category and surgical status.

All tests of significance used a 2‐sided P value <0.05. Statistical analyses were completed using Stata version 12.0 (StataCorp, College Station, TX).

RESULTS

Patient Characteristics

During the study period, 93,676 patient admissions from 53,150 unique patients resulted in the occurrence of 1,069 RRT activations, 6,305 ICU transfers, and 1,239 in‐hospital deaths. Within our sample, 40,034 patients had at least 1 inpatient record and at least 1 outpatient record. OSA diagnosis was present in 5,625 patients (10.6% of the total sample), with 4,748 patients having an OSA diagnosis code entered during a hospitalization, 2,143 with an OSA diagnosis code entered during an outpatient encounter, and 877 with both inpatient and outpatient diagnosis codes. These patients identified as having OSA contributed 12,745 (13.6%) hospital admissions.

Patients with an OSA diagnosis were more likely to be older (median age 59 years [interquartile range 4968] vs 55 years [3868]), male (49% vs 42%), overweight or obese (88% vs 62%), and more likely to carry diagnoses of diabetes (53.8% vs 25.5%), hypertension (45.3% vs 18.2%), arrhythmias (44.4% vs 26.7%), coronary artery disease (46.8% vs 23.5%), heart failure (35.8% vs 13.5%), and cerebrovascular disease (13.5% vs 8.1%) than patients without an OSA diagnosis (all comparisons significant, P < 0.001) (Table 2).

Patient Characteristics for Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; IQR, interquartile range; OSA, obstructive sleep apnea.

Age, y, median (IQR)59 (4968)55 (3868)<0.001
Female, n (%)6,514 (51%)47,202 (58%)<0.001
Race, n (%)  <0.001
White4,205 (33%)30,119 (37%) 
Black/African American7,024 (55%)38,561 (48%) 
Asian561 (4.4%)3,419 (4.2%) 
American Indian or Native Alaskan20 (0.2%)113 (0.1%) 
More than 1 race127 (1%)843 (1%) 
Race unknown808 (6%)7,876 (10%) 
Insurance status, n (%)  <0.001
Private4,484 (35%)32,467 (40%) 
Medicare/Medicaid8,201 (64%)42,208 (58%) 
Uninsured53 (0.4%)1,190 (1%) 
Unknown4 (<0.1%)16 (<0.1%) 
Location prior to wards, n (%)  <0.001
ICU1,400 (11%)8,065 (10%) 
Emergency department4,633 (36%)25,170 (31%) 
Ambulatory admission6,712 (53%)47,696 (59%) 
Body mass index, kg/m2, n (%)  <0.001
Normal (18.525)1,431 (11%)26,560 (33%) 
Underweight (<18.5)122 (1%)4,256 (5%) 
Overweight (2530)2,484 (20%)23,761 (29%) 
Obese (3040)4,959 (39%)19,132 (24%) 
Severely obese (40)3,745 (29%)7,171 (9%) 
Initial cardiac arrest risk triage score, median (IQR)4 (09)4 (09)<0.001
Underwent surgery, n (%)4,482 (35%)28,843 (36%)0.3
Comorbidities   
Number of comorbidities, median (IQR)2 (14)1 (02)<0.001
Arrhythmia5,659 (44%)21,581 (27%)<0.001
Diabetes mellitus6,855 (54%)20,641 (26%)<0.001
Hypertension5,777 (45%)14,728 (18%)<0.001
Coronary artery disease5,958 (47%)18,979 (23%)<0.001
Cerebrovascular accident1,725 (14%)6,556 (8%)<0.001
Congestive heart failure4,559 (36%)10,919 (13%)<0.001

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis
CharacteristicPatient Admissions With OSA Diagnoses, n = 12,745Patient Admissions Without OSA Diagnoses, n = 80,931P Value
  • NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea.

  • Experiencing rapid response team call, ICU transfer, or in‐hospital death.

Outcomes, n (%)   
Composite outcomea1,137 (9%)5,792 (7%)<0.001
In‐hospital death144 (1.1%)1,095 (1.4%)0.04
Rapid response team call188 (1.5%)881 (1.1%)<0.001
ICU transfer1,045 (8%)5,260 (7%)<0.001
Cardiac arrest413 (0.5%)73 (0.6%)0.36
Figure 1
Adjusted models for the association of OSA with clinical deterioration outcomes. Odds of RRT activation, intubation, ICU transfer, cardiac arrest, and in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis. Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea; RRT, rapid response team.

Sensitivity Analyses

The sensitivity analysis involving 1 randomly selected hospitalization per patient included a total of 53,150 patients. The results were similar to the main analysis, with adjusted odds of 1.01 (0.77‐1.32) for RRT activation, 0.86 (0.76‐0.96) for ICU transfer, and 0.69 (0.53‐0.89) for inpatient mortality. An additional sensitivity analysis included only patients who were not admitted to the ICU prior to their ward stay. This analysis included 84,211 hospitalizations and demonstrated similar findings, with adjusted odds of 0.70 for in‐hospital mortality (0.57‐0.87). Adjusted odds for RRT activation (OR: 1.12 [0.92‐1.37]) and ICU transfer (OR: 0.88 [0.81‐0.96] were also similar to the results of our main analysis.

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2
Adjusted models for the association of OSA with death, by surgical status and BMI. Odds of in‐hospital death for patient admissions with OSA diagnosis as compared to patient admissions without OSA diagnosis, stratified by surgical status and BMI. Abbreviations: BMI, body mass index; OSA, obstructive sleep apnea.

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m2 (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.

Acknowledgements

The authors thank Nicole Babuskow for administrative support, as well as Brian Furner and Timothy Holper for assistance with data acquisition.

Disclosures: Study concept and design: P.L., D.P.E, B.M., M.C.; acquisition of data: P.L.; analysis and interpretation of data: all authors; first drafting of the manuscript: P.L.; critical revision of the manuscript for important intellectual content: all authors; statistical analysis: P.L., F.Z., M.C.; obtained funding: D.P.E., M.C.; administrative, technical, and material support: F.Z., D.P.E.; study supervision: D.P.E, B.M., M.C.; data access and responsibility: P.L. and M.C. had full access to all the data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek and Dr. Edelson are both supported by career development awards from the National Heart, Lung, and Blood Institute (K08 HL121080 and K23 HL097157, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics.

References
  1. Peppard PE, Young T, Barnet JH, Palta M, Hagen EW, Hla KM. Increased prevalence of sleep‐disordered breathing in adults. Am J Epidemiol. 2013;177(9):10061014.
  2. Marin JM, Carrizo SJ, Vicente E, Agusti AGN. Long‐term cardiovascular outcomes in men with obstructive sleep apnoea‐hypopnoea with or without treatment with continuous positive airway pressure: an observational study. Lancet. 2005;365(9464):10461053.
  3. Peppard PE, Young T, Palta M, Skatrud J. Prospective study of the association between sleep‐disordered breathing and hypertension. N Engl J Med. 2000;342(19):13781384.
  4. Yaggi HK, Concato J, Kernan WN, Lichtman JH, Brass LM, Mohsenin V. Obstructive sleep apnea as a risk factor for stroke and death. N Engl J Med. 2005;353(19):20342041.
  5. Kendzerska T, Gershon AS, Hawker G, Leung RS, Tomlinson G. Obstructive sleep apnea and risk of cardiovascular events and all‐cause mortality: a decade‐long historical cohort study. PLoS Med. 2014;11(2):e1001599.
  6. Marshall NS, Wong KK, Liu PY, Cullen SRJ, Knuiman MW, Grunstein RR. Sleep apnea as an independent risk factor for all‐cause mortality: the Busselton Health Study. Sleep. 2008;31(8):10791085.
  7. Kaw R, Pasupuleti V, Walker E, Ramaswamy A, Foldvary‐Schafer N. Postoperative complications in patients with obstructive sleep apnea. Chest. 2012;141(2):436441.
  8. Kaw R, Chung F, Pasupuleti V, Mehta J, Gay PC, Hernandez A. Meta‐analysis of the association between obstructive sleep apnoea and postoperative outcome. Br J Anaesth. 2012;109(6):897906.
  9. Memtsoudis SG, Stundner O, Rasul R, et al. The impact of sleep apnea on postoperative utilization of resources and adverse outcomes. Anesth Analg. 2014;118(2):407418.
  10. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after bariatric surgery: analysis of the nationwide inpatient sample. Obes Surg. 2013;23(11):18421851.
  11. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after elective surgery: analysis of the nationwide inpatient sample. Chest. 2013;144:903914.
  12. Lindenauer PK, Stefan MS, Johnson KG, Priya A, Pekow PS, Rothberg MB. Prevalence, treatment and outcomes associated with obstructive sleep apnea among patients hospitalized with pneumonia. Chest. 2014;145(5):10321038.
  13. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PLoS One. 2013;8(1):e54807.
  14. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250254.
  15. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diazepam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):6569.
  16. Dolly FR, Block AJ. Effect of flurazepam on sleep‐disordered breathing and nocturnal oxygen desaturation in asymptomatic subjects. Am J Med. 1982;73(2):239243.
  17. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs*. Crit Care Med. 2012;40(7):21022108.
  18. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*. Crit Care Med. 2014;42(4):841848.
  19. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):16191636.
  20. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case‐control study. Chest. 2012;141(5):11701176.
  21. Memtsoudis SG, Bombardieri AM, Ma Y, Walz JM, Chiu YL, Mazumdar M. Mortality of patients with respiratory insufficiency and adult respiratory distress syndrome after surgery: the obesity paradox. J Intensive Care Med. 2012;27(4):306311.
  22. Bucholz EM, Rathore SS, Reid KJ, et al. Body mass index and mortality in acute myocardial infarction patients. Am J Med. 2012(8);125:796803.
  23. Murry CE, Jennings RB, Reimer KA. Preconditioning with ischemia: a delay of lethal cell injury in ischemic myocardium. Circulation. 1986;74(5):11241136.
  24. Murry CE, Richard VJ, Reimer KA, Jennings RB. Ischemic preconditioning slows energy metabolism and delays ultrastructural damage during a sustained ischemic episode. Circ Res. 1990;66(4):913931.
  25. Hu X, Yu SP, Fraser JL, et al. Transplantation of hypoxia‐preconditioned mesenchymal stem cells improves infarcted heart function via enhanced survival of implanted cells and angiogenesis. J Thorac Cardiovasc Surg. 2008;135(4):799808.
  26. Yu X, Lu C, Liu H, et al. Hypoxic preconditioning with cobalt of bone marrow mesenchymal stem cells improves cell migration and enhances therapy for treatment of ischemic acute kidney injury. PLoS One. 2013;8(5):e62703.
  27. Francis KR, Wei L. Human embryonic stem cell neural differentiation and enhanced cell survival promoted by hypoxic preconditioning. Cell Death Dis. 2010;1:e22.
  28. Kamota T, Li TS, Morikage N, et al. Ischemic pre‐conditioning enhances the mobilization and recruitment of bone marrow stem cells to protect against ischemia/reperfusion injury in the late phase. J Am Coll Cardiol. 2009;53(19):18141822.
  29. Hu X, Wei L, Taylor TM, et al. Hypoxic preconditioning enhances bone marrow mesenchymal stem cell migration via Kv2.1 channel and FAK activation. Am J Physiol Cell Physiol. 2011;301(2):C362C372.
  30. Theus MH, Wei L, Cui L, et al. In vitro hypoxic preconditioning of embryonic stem cells as a strategy of promoting cell survival and functional benefits after transplantation into the ischemic rat brain. Exp Neurol. 2008;210(2):656670.
  31. Wei L, Fraser JL, Lu ZY, Hu X, Yu SP. Transplantation of hypoxia preconditioned bone marrow mesenchymal stem cells enhances angiogenesis and neurogenesis after cerebral ischemia in rats. Neurobiol Dis. 2012;46(3):635645.
  32. Kharbanda RK, Nielsen TT, Redington AN. Translation of remote ischaemic preconditioning into clinical practice. Lancet. 2009;374(9700):15571565.
  33. Schmidt MR, Pryds K, Bøtker HE. Novel adjunctive treatments of myocardial infarction. World J Cardiol. 2014;6(6):434443.
  34. Ara J, Montpellier S. Hypoxic‐preconditioning enhances the regenerative capacity of neural stem/progenitors in subventricular zone of newborn piglet brain. Stem Cell Res. 2013;11(2):669686.
  35. Foster GP, Giri PC, Rogers DM, Larson SR, Anholm JD. Ischemic preconditioning improves oxygen saturation and attenuates hypoxic pulmonary vasoconstriction at high altitude. High Alt Med Biol. 2014;15(2):155161.
  36. Jean‐St‐Michel E, Manlhiot C, Li J, et al. Remote preconditioning improves maximal performance in highly trained athletes. Med Sci Sports Exerc. 2011;43(7):12801286.
  37. Durán J, Esnaola S, Rubio R, Iztueta Á. Obstructive sleep apnea‐hypopnea and related clinical features in a population‐based sample of subjects aged 30 to 70 yr. Am J Respir Crit Care Med. 2001;163(3 pt 1):685689.
  38. Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep‐disordered breathing among middle‐aged adults. N Engl J Med. 1993;328(17):12301235.
  39. Young T, Peppard PE, Gottlieb DJ. Epidemiology of obstructive sleep apnea: a population health perspective. Am J Respir Crit Care Med. 2002;165(9):12171239.
  40. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
References
  1. Peppard PE, Young T, Barnet JH, Palta M, Hagen EW, Hla KM. Increased prevalence of sleep‐disordered breathing in adults. Am J Epidemiol. 2013;177(9):10061014.
  2. Marin JM, Carrizo SJ, Vicente E, Agusti AGN. Long‐term cardiovascular outcomes in men with obstructive sleep apnoea‐hypopnoea with or without treatment with continuous positive airway pressure: an observational study. Lancet. 2005;365(9464):10461053.
  3. Peppard PE, Young T, Palta M, Skatrud J. Prospective study of the association between sleep‐disordered breathing and hypertension. N Engl J Med. 2000;342(19):13781384.
  4. Yaggi HK, Concato J, Kernan WN, Lichtman JH, Brass LM, Mohsenin V. Obstructive sleep apnea as a risk factor for stroke and death. N Engl J Med. 2005;353(19):20342041.
  5. Kendzerska T, Gershon AS, Hawker G, Leung RS, Tomlinson G. Obstructive sleep apnea and risk of cardiovascular events and all‐cause mortality: a decade‐long historical cohort study. PLoS Med. 2014;11(2):e1001599.
  6. Marshall NS, Wong KK, Liu PY, Cullen SRJ, Knuiman MW, Grunstein RR. Sleep apnea as an independent risk factor for all‐cause mortality: the Busselton Health Study. Sleep. 2008;31(8):10791085.
  7. Kaw R, Pasupuleti V, Walker E, Ramaswamy A, Foldvary‐Schafer N. Postoperative complications in patients with obstructive sleep apnea. Chest. 2012;141(2):436441.
  8. Kaw R, Chung F, Pasupuleti V, Mehta J, Gay PC, Hernandez A. Meta‐analysis of the association between obstructive sleep apnoea and postoperative outcome. Br J Anaesth. 2012;109(6):897906.
  9. Memtsoudis SG, Stundner O, Rasul R, et al. The impact of sleep apnea on postoperative utilization of resources and adverse outcomes. Anesth Analg. 2014;118(2):407418.
  10. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after bariatric surgery: analysis of the nationwide inpatient sample. Obes Surg. 2013;23(11):18421851.
  11. Mokhlesi B, Hovda MD, Vekhter B, Arora VM, Chung F, Meltzer DO. Sleep‐disordered breathing and postoperative outcomes after elective surgery: analysis of the nationwide inpatient sample. Chest. 2013;144:903914.
  12. Lindenauer PK, Stefan MS, Johnson KG, Priya A, Pekow PS, Rothberg MB. Prevalence, treatment and outcomes associated with obstructive sleep apnea among patients hospitalized with pneumonia. Chest. 2014;145(5):10321038.
  13. Doufas AG, Tian L, Padrez KA, et al. Experimental pain and opioid analgesia in volunteers at high risk for obstructive sleep apnea. PLoS One. 2013;8(1):e54807.
  14. Gislason T, Almqvist M, Boman G, Lindholm CE, Terenius L. Increased CSF opioid activity in sleep apnea syndrome. Regression after successful treatment. Chest. 1989;96(2):250254.
  15. Koga Y, Sato S, Sodeyama N, et al. Comparison of the relaxant effects of diazepam, flunitrazepam and midazolam on airway smooth muscle. Br J Anaesth. 1992;69(1):6569.
  16. Dolly FR, Block AJ. Effect of flurazepam on sleep‐disordered breathing and nocturnal oxygen desaturation in asymptomatic subjects. Am J Med. 1982;73(2):239243.
  17. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs*. Crit Care Med. 2012;40(7):21022108.
  18. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*. Crit Care Med. 2014;42(4):841848.
  19. Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest. 1991;100(6):16191636.
  20. Churpek MM, Yuen TC, Huber MT, Park SY, Hall JB, Edelson DP. Predicting cardiac arrest on the wards: a nested case‐control study. Chest. 2012;141(5):11701176.
  21. Memtsoudis SG, Bombardieri AM, Ma Y, Walz JM, Chiu YL, Mazumdar M. Mortality of patients with respiratory insufficiency and adult respiratory distress syndrome after surgery: the obesity paradox. J Intensive Care Med. 2012;27(4):306311.
  22. Bucholz EM, Rathore SS, Reid KJ, et al. Body mass index and mortality in acute myocardial infarction patients. Am J Med. 2012(8);125:796803.
  23. Murry CE, Jennings RB, Reimer KA. Preconditioning with ischemia: a delay of lethal cell injury in ischemic myocardium. Circulation. 1986;74(5):11241136.
  24. Murry CE, Richard VJ, Reimer KA, Jennings RB. Ischemic preconditioning slows energy metabolism and delays ultrastructural damage during a sustained ischemic episode. Circ Res. 1990;66(4):913931.
  25. Hu X, Yu SP, Fraser JL, et al. Transplantation of hypoxia‐preconditioned mesenchymal stem cells improves infarcted heart function via enhanced survival of implanted cells and angiogenesis. J Thorac Cardiovasc Surg. 2008;135(4):799808.
  26. Yu X, Lu C, Liu H, et al. Hypoxic preconditioning with cobalt of bone marrow mesenchymal stem cells improves cell migration and enhances therapy for treatment of ischemic acute kidney injury. PLoS One. 2013;8(5):e62703.
  27. Francis KR, Wei L. Human embryonic stem cell neural differentiation and enhanced cell survival promoted by hypoxic preconditioning. Cell Death Dis. 2010;1:e22.
  28. Kamota T, Li TS, Morikage N, et al. Ischemic pre‐conditioning enhances the mobilization and recruitment of bone marrow stem cells to protect against ischemia/reperfusion injury in the late phase. J Am Coll Cardiol. 2009;53(19):18141822.
  29. Hu X, Wei L, Taylor TM, et al. Hypoxic preconditioning enhances bone marrow mesenchymal stem cell migration via Kv2.1 channel and FAK activation. Am J Physiol Cell Physiol. 2011;301(2):C362C372.
  30. Theus MH, Wei L, Cui L, et al. In vitro hypoxic preconditioning of embryonic stem cells as a strategy of promoting cell survival and functional benefits after transplantation into the ischemic rat brain. Exp Neurol. 2008;210(2):656670.
  31. Wei L, Fraser JL, Lu ZY, Hu X, Yu SP. Transplantation of hypoxia preconditioned bone marrow mesenchymal stem cells enhances angiogenesis and neurogenesis after cerebral ischemia in rats. Neurobiol Dis. 2012;46(3):635645.
  32. Kharbanda RK, Nielsen TT, Redington AN. Translation of remote ischaemic preconditioning into clinical practice. Lancet. 2009;374(9700):15571565.
  33. Schmidt MR, Pryds K, Bøtker HE. Novel adjunctive treatments of myocardial infarction. World J Cardiol. 2014;6(6):434443.
  34. Ara J, Montpellier S. Hypoxic‐preconditioning enhances the regenerative capacity of neural stem/progenitors in subventricular zone of newborn piglet brain. Stem Cell Res. 2013;11(2):669686.
  35. Foster GP, Giri PC, Rogers DM, Larson SR, Anholm JD. Ischemic preconditioning improves oxygen saturation and attenuates hypoxic pulmonary vasoconstriction at high altitude. High Alt Med Biol. 2014;15(2):155161.
  36. Jean‐St‐Michel E, Manlhiot C, Li J, et al. Remote preconditioning improves maximal performance in highly trained athletes. Med Sci Sports Exerc. 2011;43(7):12801286.
  37. Durán J, Esnaola S, Rubio R, Iztueta Á. Obstructive sleep apnea‐hypopnea and related clinical features in a population‐based sample of subjects aged 30 to 70 yr. Am J Respir Crit Care Med. 2001;163(3 pt 1):685689.
  38. Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep‐disordered breathing among middle‐aged adults. N Engl J Med. 1993;328(17):12301235.
  39. Young T, Peppard PE, Gottlieb DJ. Epidemiology of obstructive sleep apnea: a population health perspective. Am J Respir Crit Care Med. 2002;165(9):12171239.
  40. Churpek MM, Yuen TC, Edelson DP. Risk stratification of hospitalized patients on the wards. Chest. 2013;143(6):17581765.
Issue
Journal of Hospital Medicine - 10(9)
Issue
Journal of Hospital Medicine - 10(9)
Page Number
592-598
Page Number
592-598
Publications
Publications
Article Type
Display Headline
Obstructive sleep apnea and adverse outcomes in surgical and nonsurgical patients on the wards
Display Headline
Obstructive sleep apnea and adverse outcomes in surgical and nonsurgical patients on the wards
Sections
Article Source

© 2015 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Matthew M. Churpek, MD, Section of Pulmonary and Critical Care, University of Chicago, 5841 S Maryland Avenue, MC 6076, Chicago, IL 60637; Telephone: 773‐702‐1092; Fax: 773‐702‐6500; E‐mail: matthew.churpek@uchospitals.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

Optimal Rapid Response System Bundle

Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
In search of the optimal rapid response system bundle

The theory behind rapid response teams (RRTs), namely to provide critical care resources to patients with clinical deterioration on the wards, is such common sense that failure to do so seems unethical. This idea, combined with evidence that many cardiac arrests on the wards are predictable and potentially preventable events, led to the proliferation of RRTs across the country and a Joint Commission mandate.[1] However, data from clinical trials have failed to consistently confirm the value of these teams, likely a product of the wide variability in implementation practices across institutions.[2]

In this issue of the Journal of Hospital Medicine, Davis and colleagues demonstrate improvements in both mortality and cardiac arrest rates outside the intensive care unit (ICU) following implementation of their rapid response system in 2 hospitals.[3] Although several other studies have shown similar results, what makes this implementation unique is the bundle approach that included proactive rounding by the charge nurse from each unit, annual focused training of team members and staff, and an integrated, continuous, quality‐improvement feedback loop. Bundles are common in successful quality‐improvement work, but can be challenging for deciphering which of the individual components are driving the results, leaving readers to venture an educated guess. In the current bundle, the novel use of the charge nurse has some significant appeal as a candidate primary driver of the impact, because it likely had 2 distinct actions: (1) proactive rounding and (2) promoting a culture change, both of which are well supported in the literature.4,5

Several studies, including this one, have demonstrated a dose‐response association between the number of RRT activations and patient outcomes, with a low number of RRT activations deemed a major contributor to the neutral results of the large multicenter, randomized, controlled MERIT trial.[6, 7] Additionally, delays in treatment and transfer to the ICU for unstable patients are known to increase mortality.[8] One way to increase the number of patients seen by the RRT and decrease activation delays is by instituting proactive rounding by the team on high‐risk patients. This was the strategy employed in a landmark ward‐randomized trial by Priestley and colleagues, which demonstrated a significant improvement in mortality from proactive rounding on patients deemed to be at high risk of clinical deterioration as calculated by an early warning score or due to caregiver concern.4

Identification of at‐risk patients for proactive rounding can be accomplished with gestalt, as was done by the charge nurse in the current study, or using specific individual criteria such as recent discharge from an ICU. Alternatively, this can be accomplished using composite vital signbased risk scores, such as the Modified Early Warning Score (MEWS).[9] Recently, several newer algorithms that integrate vital signs, laboratory data, and demographics have been shown to outperform the MEWS.[10, 11] Such systems promise an exciting age of real‐time computer‐generated risk stratification, with the ability to automate and standardize the selection of patients for proactive rounding across institutions.

Interestingly, the selection of the charge nurse, rather than someone who did not reside on the unit, to conduct the surveillance rounds likely had another benefit: expediting and facilitating the culture change necessary for a successful implementation. The integration of the charge nurse into the RRT likely led to a local reinforcement of important cultural changes that were already happening at the institutional level. It is clear that culture change is essential in any quality improvement endeavor, and previous literature on RRTs supports this notion.[5]

Rapid response systems are complex and include the activation criteria, team composition and training, and an administrative component. A multifaceted, bundled approach is likely to be required for success. Furthermore, regardless of what risk stratification criteria are used, proactive rounding on high‐risk patients is likely to increase the yield. Utilizing the charge nurse in that effort is a creative use of a preexisting local resource and is worthy of future study.

Disclosures: Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from CHEST for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients, and Dr. Edelson has an ownership interest in Quant HC (Chicago, IL), which seeks to commercialize those algorithms.

References
  1. Jones DA, DeVita MA, Bellomo R. Rapid‐response teams. N Engl J Med. 2011;365(2):139146.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Davis DP, Aguilar SA, Graham PG, et al. A novel configuration of a traditional rapid response team decreases non‐ICU arrests and overall hospital mortality. J Hosp Med. 2015;10(6):352357
  4. Priestley G, Watson W, Rashidian A, et al. Introducing Critical Care Outreach: a ward‐randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004;30(7):13981404.
  5. Stevens J, Johansson A, Lennes I, Hsu D, Tess A, Howell M. Long‐term culture change related to rapid response system implementation. Med Educ. 2014;48(12):12111219.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Jones D, Bellomo R, DeVita MA. Effectiveness of the Medical Emergency Team: the importance of dose. Crit Care. 2009;13(5):313.
  8. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  9. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  10. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  11. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388395.
Article PDF
Issue
Journal of Hospital Medicine - 10(6)
Publications
Page Number
411-411
Sections
Article PDF
Article PDF

The theory behind rapid response teams (RRTs), namely to provide critical care resources to patients with clinical deterioration on the wards, is such common sense that failure to do so seems unethical. This idea, combined with evidence that many cardiac arrests on the wards are predictable and potentially preventable events, led to the proliferation of RRTs across the country and a Joint Commission mandate.[1] However, data from clinical trials have failed to consistently confirm the value of these teams, likely a product of the wide variability in implementation practices across institutions.[2]

In this issue of the Journal of Hospital Medicine, Davis and colleagues demonstrate improvements in both mortality and cardiac arrest rates outside the intensive care unit (ICU) following implementation of their rapid response system in 2 hospitals.[3] Although several other studies have shown similar results, what makes this implementation unique is the bundle approach that included proactive rounding by the charge nurse from each unit, annual focused training of team members and staff, and an integrated, continuous, quality‐improvement feedback loop. Bundles are common in successful quality‐improvement work, but can be challenging for deciphering which of the individual components are driving the results, leaving readers to venture an educated guess. In the current bundle, the novel use of the charge nurse has some significant appeal as a candidate primary driver of the impact, because it likely had 2 distinct actions: (1) proactive rounding and (2) promoting a culture change, both of which are well supported in the literature.4,5

Several studies, including this one, have demonstrated a dose‐response association between the number of RRT activations and patient outcomes, with a low number of RRT activations deemed a major contributor to the neutral results of the large multicenter, randomized, controlled MERIT trial.[6, 7] Additionally, delays in treatment and transfer to the ICU for unstable patients are known to increase mortality.[8] One way to increase the number of patients seen by the RRT and decrease activation delays is by instituting proactive rounding by the team on high‐risk patients. This was the strategy employed in a landmark ward‐randomized trial by Priestley and colleagues, which demonstrated a significant improvement in mortality from proactive rounding on patients deemed to be at high risk of clinical deterioration as calculated by an early warning score or due to caregiver concern.4

Identification of at‐risk patients for proactive rounding can be accomplished with gestalt, as was done by the charge nurse in the current study, or using specific individual criteria such as recent discharge from an ICU. Alternatively, this can be accomplished using composite vital signbased risk scores, such as the Modified Early Warning Score (MEWS).[9] Recently, several newer algorithms that integrate vital signs, laboratory data, and demographics have been shown to outperform the MEWS.[10, 11] Such systems promise an exciting age of real‐time computer‐generated risk stratification, with the ability to automate and standardize the selection of patients for proactive rounding across institutions.

Interestingly, the selection of the charge nurse, rather than someone who did not reside on the unit, to conduct the surveillance rounds likely had another benefit: expediting and facilitating the culture change necessary for a successful implementation. The integration of the charge nurse into the RRT likely led to a local reinforcement of important cultural changes that were already happening at the institutional level. It is clear that culture change is essential in any quality improvement endeavor, and previous literature on RRTs supports this notion.[5]

Rapid response systems are complex and include the activation criteria, team composition and training, and an administrative component. A multifaceted, bundled approach is likely to be required for success. Furthermore, regardless of what risk stratification criteria are used, proactive rounding on high‐risk patients is likely to increase the yield. Utilizing the charge nurse in that effort is a creative use of a preexisting local resource and is worthy of future study.

Disclosures: Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from CHEST for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients, and Dr. Edelson has an ownership interest in Quant HC (Chicago, IL), which seeks to commercialize those algorithms.

The theory behind rapid response teams (RRTs), namely to provide critical care resources to patients with clinical deterioration on the wards, is such common sense that failure to do so seems unethical. This idea, combined with evidence that many cardiac arrests on the wards are predictable and potentially preventable events, led to the proliferation of RRTs across the country and a Joint Commission mandate.[1] However, data from clinical trials have failed to consistently confirm the value of these teams, likely a product of the wide variability in implementation practices across institutions.[2]

In this issue of the Journal of Hospital Medicine, Davis and colleagues demonstrate improvements in both mortality and cardiac arrest rates outside the intensive care unit (ICU) following implementation of their rapid response system in 2 hospitals.[3] Although several other studies have shown similar results, what makes this implementation unique is the bundle approach that included proactive rounding by the charge nurse from each unit, annual focused training of team members and staff, and an integrated, continuous, quality‐improvement feedback loop. Bundles are common in successful quality‐improvement work, but can be challenging for deciphering which of the individual components are driving the results, leaving readers to venture an educated guess. In the current bundle, the novel use of the charge nurse has some significant appeal as a candidate primary driver of the impact, because it likely had 2 distinct actions: (1) proactive rounding and (2) promoting a culture change, both of which are well supported in the literature.4,5

Several studies, including this one, have demonstrated a dose‐response association between the number of RRT activations and patient outcomes, with a low number of RRT activations deemed a major contributor to the neutral results of the large multicenter, randomized, controlled MERIT trial.[6, 7] Additionally, delays in treatment and transfer to the ICU for unstable patients are known to increase mortality.[8] One way to increase the number of patients seen by the RRT and decrease activation delays is by instituting proactive rounding by the team on high‐risk patients. This was the strategy employed in a landmark ward‐randomized trial by Priestley and colleagues, which demonstrated a significant improvement in mortality from proactive rounding on patients deemed to be at high risk of clinical deterioration as calculated by an early warning score or due to caregiver concern.4

Identification of at‐risk patients for proactive rounding can be accomplished with gestalt, as was done by the charge nurse in the current study, or using specific individual criteria such as recent discharge from an ICU. Alternatively, this can be accomplished using composite vital signbased risk scores, such as the Modified Early Warning Score (MEWS).[9] Recently, several newer algorithms that integrate vital signs, laboratory data, and demographics have been shown to outperform the MEWS.[10, 11] Such systems promise an exciting age of real‐time computer‐generated risk stratification, with the ability to automate and standardize the selection of patients for proactive rounding across institutions.

Interestingly, the selection of the charge nurse, rather than someone who did not reside on the unit, to conduct the surveillance rounds likely had another benefit: expediting and facilitating the culture change necessary for a successful implementation. The integration of the charge nurse into the RRT likely led to a local reinforcement of important cultural changes that were already happening at the institutional level. It is clear that culture change is essential in any quality improvement endeavor, and previous literature on RRTs supports this notion.[5]

Rapid response systems are complex and include the activation criteria, team composition and training, and an administrative component. A multifaceted, bundled approach is likely to be required for success. Furthermore, regardless of what risk stratification criteria are used, proactive rounding on high‐risk patients is likely to increase the yield. Utilizing the charge nurse in that effort is a creative use of a preexisting local resource and is worthy of future study.

Disclosures: Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080) and has received honoraria from CHEST for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD.P0535US.P2) for risk stratification algorithms for hospitalized patients, and Dr. Edelson has an ownership interest in Quant HC (Chicago, IL), which seeks to commercialize those algorithms.

References
  1. Jones DA, DeVita MA, Bellomo R. Rapid‐response teams. N Engl J Med. 2011;365(2):139146.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Davis DP, Aguilar SA, Graham PG, et al. A novel configuration of a traditional rapid response team decreases non‐ICU arrests and overall hospital mortality. J Hosp Med. 2015;10(6):352357
  4. Priestley G, Watson W, Rashidian A, et al. Introducing Critical Care Outreach: a ward‐randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004;30(7):13981404.
  5. Stevens J, Johansson A, Lennes I, Hsu D, Tess A, Howell M. Long‐term culture change related to rapid response system implementation. Med Educ. 2014;48(12):12111219.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Jones D, Bellomo R, DeVita MA. Effectiveness of the Medical Emergency Team: the importance of dose. Crit Care. 2009;13(5):313.
  8. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  9. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  10. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  11. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388395.
References
  1. Jones DA, DeVita MA, Bellomo R. Rapid‐response teams. N Engl J Med. 2011;365(2):139146.
  2. Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  3. Davis DP, Aguilar SA, Graham PG, et al. A novel configuration of a traditional rapid response team decreases non‐ICU arrests and overall hospital mortality. J Hosp Med. 2015;10(6):352357
  4. Priestley G, Watson W, Rashidian A, et al. Introducing Critical Care Outreach: a ward‐randomised trial of phased introduction in a general hospital. Intensive Care Med. 2004;30(7):13981404.
  5. Stevens J, Johansson A, Lennes I, Hsu D, Tess A, Howell M. Long‐term culture change related to rapid response system implementation. Med Educ. 2014;48(12):12111219.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Jones D, Bellomo R, DeVita MA. Effectiveness of the Medical Emergency Team: the importance of dose. Crit Care. 2009;13(5):313.
  8. Young MP, Gooder VJ, McBride K, James B, Fisher ES. Inpatient transfers to the intensive care unit: delays are associated with increased mortality and morbidity. J Gen Intern Med. 2003;18(2):7783.
  9. Subbe CP, Kruger M, Rutherford P, Gemmel L. Validation of a modified Early Warning Score in medical admissions. QJM. 2001;94(10):521526.
  10. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649655.
  11. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388395.
Issue
Journal of Hospital Medicine - 10(6)
Issue
Journal of Hospital Medicine - 10(6)
Page Number
411-411
Page Number
411-411
Publications
Publications
Article Type
Display Headline
In search of the optimal rapid response system bundle
Display Headline
In search of the optimal rapid response system bundle
Sections
Article Source
© 2015 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, University of Chicago Medical Center, Section of Hospital Medicine, 5841 South Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773‐834‐2191; Fax: 773‐795‐7398; E‐mail: dperes@uchicago.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media

In‐hospital CPR Practices

Article Type
Changed
Sun, 05/21/2017 - 14:13
Display Headline
Hospital cardiac arrest resuscitation practice in the United States: A nationally representative survey

An estimated 200,000 adult patients suffer cardiac arrest in US hospitals each year, of which <20% survive to hospital discharge.[1, 2] Patient survival from in‐hospital cardiac arrest (IHCA), however, varies widely across hospitals, and may be partly attributed to differences in hospital practices.[3, 4, 5] Although there are data to support specific patient‐level practices in the hospital, such as delivery of electrical shock for ventricular fibrillation within 2 minutes of onset of the lethal rhythm,[6] little is known about in‐hospital systems‐level factors. Similar to patient‐level practices, some organizational and systems level practices are supported by international consensus and guideline recommendations.[7, 8] However, the adoption of these practices is poorly understood. As such, we sought to gain a better understanding of current US hospital practices with regard to IHCA and resuscitation with the hopes of identifying potential targets for improvement in quality and outcomes.

METHODS

We conducted a nationally representative mail survey between May 2011 and November 2011, targeting a stratified random sample of 1000 hospitals. We utilized the US Acute‐Care Hospitals (FY2008) database from the American Hospital Association to determine the total population of 3809 community hospitals (ie, nonfederal government, nonpsychiatric, and nonlong‐term care hospitals).[9] This included general medical and surgical, surgical, cancer, heart, orthopedic, and children's hospitals. These hospitals were stratified into tertiles by annual in‐patient days and teaching status (major, minor, nonteaching), from which our sample was randomly selected (Table 1). We identified each hospital's cardiopulmonary resuscitation (CPR) committee (sometimes known as code committee, code blue committee, or cardiac arrest committee) chair or chief medical/quality officer, to whom the paper‐based survey was addressed, with instructions to forward to the most appropriate person if someone other than the recipient. This study was evaluated by the University of Chicago institutional review board and deemed exempt from further review.

Figure 1
Hospital responders to in‐hospital resuscitations by institution type and level of participation. Bars represent the percent of hospitals reporting usual resuscitation responders in their hospitals, stratified by the teaching status of the hospital. Each bar is further subdivided by the likelihood of that provider to lead the resuscitation.

Survey

The survey content was developed by the study investigators and iteratively adapted by consensus and beta testing to require approximately 10 minutes to complete. Questions were edited and formatted by the University of Chicago Survey Lab (Chicago, IL) to be more precise and generalizable. Surveys were mailed in May 2011 and resent twice to nonresponders. A $10 incentive was included in the second mailing. When more than 1 response from a hospital was received, the more complete survey was used, or if equally complete, the responses were combined. All printing, mailing, receipt control, and data entry were performed by the University of Chicago Survey Lab, and data entry was double‐keyed to ensure accuracy.

Response rate was calculated based on the American Association for Public Opinion Research standard response rate formula.[10] It was assumed that the portion of nonresponding cases were ineligible at the same rate of cases for which eligibility was determined. A survey was considered complete if at least 75% of individual questions contained a valid response, partially complete if at least 40% but less than 75% of questions contained a valid response, and a nonresponse if less than 40% was completed. Nonresponses were excluded from the analysis.

Statistical Analysis

Analyses were performed using a statistical software application (Stata version 11.0; StataCorp, College Station, TX). Descriptive statistics were calculated and presented as number (%) or median (interquartile range). A [2] statistic was used to assess bias in response rate. We determined a priori 2 indicators of resource allocation (availability of a CPR committee and dedicated personnel for resuscitation quality improvement) and tested their association with quality improvement initiatives, using logistic regression to adjust for hospital teaching status and number of admissions as potential confounders. All tests of significance used a 2‐sided P<0.05.

RESULTS

Responses were received from 439 hospitals (425 complete and 14 partially complete), yielding a response rate of 44%. One subject ID was removed from the survey and could not be identified, so it was excluded from any analyses. Hospital demographics were similar between responders and nonresponders (P=0.50) (Table 1). Respondents who filled out the surveys included chief medical/quality officers (n=143 [33%]), chairs of CPR committees (n=64 [15%]), members of CPR committees (n=29 [7%]), chiefs of staff (n=33 [8%]), resuscitation officers/nurses (n=27 [6%]), chief nursing officers (n=13 [3%]), and others (n=131 [30%]).

Stratified Response Rates by Hospital Volume and Teaching Status
Teaching StatusAnnual Inpatient DaysTotal
<17,69517,695‐52,500>52,500
  • NOTE: Results are shown as number of respondents/total sampled (%).

Major1/2 (50)1/8 (13)40/82 (49)42/92 (46)
Minor13/39 (33)40/89 (45)62/133 (47)115/261 (44)
Nonteaching141/293 (48)100/236 (42)40/118 (34)281/647 (43)
Total156/335 (47)143/335 (43)145/336 (43)438/1,000 (44)

Table 2 summarizes structure, equipment, quality improvement, and pre‐ and postarrest practices across the hospitals. Of note, 77% of hospitals (n=334) reported having a predesignated, dedicated code team, and 66% (n=281) reported standardized defibrillator make and model throughout their hospital. However, less than one‐third of hospitals utilized any CPR assist technology (eg, CPR quality sensor or mechanical CPR device). The majority of hospitals reported having a rapid response team (RRT) (n=391 [91%]). Although a therapeutic hypothermia protocol for postarrest care was in place in over half of hospitals (n=252 [58%]), utilization of hypothermia for patients with return of spontaneous circulation was infrequent.

In‐hospital Resuscitation Structure and Practices
 Value2010 AHA Guidelines
  • NOTE: Results are shown as total (%) unless otherwise indicated. Percentages were adjusted by excluding missing responses. Abbreviations: AED, automatic external defibrillator; AHA, American Heart Association; CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; IQR, interquartile range; LOE, level of evidence; PA, public address; QI, quality improvement; ROSC, return of spontaneous circulation; RRT, rapid response team; TH, therapeutic hypothermia

  • These categories are not mutually exclusive

  • Recommended or supported in 2005 guidelines

  • May be considered for use in specific settings by properly trained personnel

  • Supported in the guidelines without official class recommendation.

Structure  
Existing CPR committee270 (66) 
CPR chair  
Physician only129 (48) 
Nurse only90 (34) 
Nurse/physician co‐chair31 (12) 
Other17 (6) 
Clinical specialty of chaira  
Pulmonary/critical care79 (35) 
Emergency medicine71 (31) 
Anesthesia/critical care43 (19) 
Cardiology38 (17) 
Other32 (14) 
Hospital medicine23 (10) 
Predetermined cardiac arrest team structure334 (77) 
Notifications of respondersa  
Hospital‐wide PA system406 (93) 
Pager/calls to individuals230 (53) 
Local alarm49 (11) 
Equipment  
AEDs used as primary defibrillator by location  
High‐acuity inpatient areas69 (16) 
Low‐acuity inpatient areas109 (26) 
Outpatient areas206 (51)Class IIb, LOE Cb
Public areas263 (78)Class IIb, LOE Cb
Defibrillator throughout hospital  
Same brand and model281 (66) 
Same brand, different models93 (22) 
Different brands54 (13) 
CPR assist technology useda  
None291 (70) 
Capnography106 (25)Class IIb, LOE Cb
Mechanical CPR25 (6)Class IIb, LOE B/Cbc
Feedback device17 (4)Class IIa, LOE B
Quality improvement  
IHCA tracked336 (82)Supportedbd
Data reviewed Supportedbd
Data not tracked/never reviewed85 (20) 
Intermittently53 (12) 
Routinely287 (68) 
Routine cardiac arrest case reviews/debriefing149 (34)Class IIa, LOE C
Dedicated staff to resuscitation QI196 (49) 
Full‐time equivalent staffing, median (IQR)0.5 (0.251.2) 
Routine simulated resuscitation training268 (62) 
Pre‐ and postarrest measures  
Hospitals with RRT391 (91)Class I, LOE Cb
Formal RRT‐specific training  
Never50 (14) 
Once110 (30) 
Recurrent163 (45) 
TH protocol/order set in place252 (58) 
Percent of patients with ROSC receiving TH Class IIb, LOE Bb
<5%309 (74) 
5%25%68 (16) 
26%50%11 (3) 
51%75%10 (2) 
>75%18 (4) 

Hospitals reported that routine responders to IHCA events included respiratory therapists (n=414 [95%]), critical care nurses (n=406 [93%]), floor nurses (n=396 [90%]), attending physicians (n=392 [89%]), physician trainees (n=162 [37%]), and pharmacists (n=210 [48%]). Figure 1 shows the distribution of responders and team leaders by hospital type. Of the nonteaching hospitals, attending‐level physicians were likely to respond at 94% (265/281) and routinely lead the resuscitations at 84% (236/281), whereas, of major teaching hospitals, attending physicians were only likely to respond at 71% (30/42) and routinely lead at 19% (8/42).

Two‐thirds of the hospitals had a CPR committee (n=270 [66%]), and 196 (49%) had some staff time dedicated to resuscitation quality improvement. Hospitals with a specific committee dedicated to resuscitation and/or dedicated staff for resuscitation quality improvement were more likely to routinely track cardiac arrest data (odds ratio [OR]: 3.64, 95% confidence interval [CI]: 2.056.47 and OR: 2.02, 95% CI: 1.16‐3.54, respectively) and review the data (OR: 2.67, 95% CI: 1.45‐4.92 and OR: 2.18, 95% CI: 1.22‐3.89, respectively), after adjusting for teaching status and hospital size. These hospitals were also more likely to engage in simulation training and debriefing (Table 3).

Correlation Between Resource Availability and Quality Improvement Practices
 CPR Committee, n=406Dedicated QI Staff, n=398
  • NOTE: Logistic regression adjusting for hospital size and teaching status was performed. All results are shown as odds ratio (95% confidence interval)

  • Abbreviations: CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; QI, quality improvement.

IHCA tracking3.64 (2.056.47)2.02 (1.16‐3.54)
Routinely review2.67 (1.45‐4.92)2.18 (1.22‐3.89)
Simulation training2.63 (1.66‐4.18)1.89 (1.24‐2.89)
Debriefing3.19 (1.89‐5.36)2.14 (1.39‐3.32)

Ninety percent (n=391) of respondents agreed that there is room for improvement in resuscitation practice at my hospital, and 70% (n=302) agreed that improved resuscitation would translate into improved patient outcomes. Overall, 78% (n=338) cited at least 1 barrier to improved resuscitation quality, of which the lack of adequate training (n=233 [54%]) and the lack of an appropriate champion (n=230 [53%]) were the most common. In subgroup analysis, nonteaching hospitals were significantly more likely to report the lack of a champion than their teaching counterparts (P=0.001) (Figure 2). In addition, we analyzed the data by hospitals that reported lack of a champion was not a barrier and compared them to those for whom it was, and found significantly higher adherence across all the measures in Table 2 supported by the 2010 guidelines, with the exception of real‐time feedback (data not shown).

Figure 2
Barriers to resuscitation quality improvement by institution type. Bars represent the percent of responders reporting specific perceived barriers to resuscitation quality improvement at their hospital, stratified by the teaching status of the hospital.

DISCUSSION

In this nationally representative sample of hospitals, we found considerable variability in cardiac arrest and resuscitation structures and processes, suggesting potential areas to target for improvement. Some practices, including use of RRTs and defibrillator standardization, were fairly routine, whereas others, such as therapeutic hypothermia and CPR assist technology, were rarely utilized. Quality initiatives, such as data tracking and review, simulation training, and debriefing were variable.

Several factors likely contribute to the variable implementation of evidence‐based practices. Guidelines alone have been shown to have little impact on practice by physicians in general.[11] This is supported by the lack of correlation we found between the presence, absence or strength of specific American Heart Association (AHA) emergency cardiovascular care treatment recommendations and the percent of hospitals reporting performing that measure. It is possible that other factors, such as a lack of familiarity or agreement with those guidelines, or the presence of external barriers, may be contributing.[12, 13] Specifically, the importance of a clinical champion was supported by our finding that hospitals reporting lack of a champion as a barrier were less likely to be adherent with guidelines. However, because the study did not directly test the impact of a champion, we wanted to be careful to avoid overstating or editorializing our results.

Some of the variability may also be related to the resource intensiveness of the practice. Routine simulation training and debriefing interventions, for example, are time intensive and require trained personnel to institute. That may explain the correlation we noted between these practices and the presence of CPR committee and dedicated personnel. The use of dedicated personnel was rare in this study, with less than half of respondents reporting any dedicated staff and a median of 0.5 full‐time equivalents for those reporting positively. This is in stark contrast to the routine use of resuscitation officers (primarily nurses dedicated to overseeing resuscitation practices and education at the hospital) in the United Kingdom.[14] Such a resuscitation officer model adopted by US hospitals could improve the quality and intensity of resuscitation care approaches.

Particularly surprising was the high rate of respondents (70%) reporting that they do not utilize any CPR assist technology. In the patient who does not have an arterial line, use of quantitative capnography is the best measure of cardiac output during cardiac arrest, yet only one‐quarter of hospitals reported using it, with no discrepancy between hospital type or size. A recent summit of national resuscitation experts expounded on the AHA guidelines suggesting that end‐tidal carbon dioxide should be used in all arrests to guide the quality of CPR with a goal value of >20.[8] Similarly, CPR feedback devices have an even higher level of evidence recommendation in the 2010 AHA guidelines than capnography, yet only 4% of hospitals reported utilizing them. Although it is true that introducing these CPR assist technologies into a hospital would require some effort on the part of hospital leadership, it is important to recognize the potential role such devices might play in the larger context of a resuscitation quality program to optimize clinical outcomes from IHCA.

Several differences were noted between hospitals based on teaching status. Although all hospitals were more likely to rely on physicians to lead resuscitations, nonteaching hospitals were more likely to report routine leadership by nurses and pharmacists. Nonteaching hospitals were also less likely to have a CPR committee, even after adjusting for hospital size. In addition, these hospitals were also more likely to report the lack of a clinical champion as a barrier to quality improvement.

There were several limitations to this study. First, this was a descriptive survey that was not tied to outcomes. As such, we are unable to draw conclusions about which practices correlate with decreased incidence of cardiac arrest and improved survival. Second, this was an optional survey with a somewhat limited response rate. Even though the characteristics of the nonresponding hospitals were similar to the responding hospitals, we cannot rule out the possibility that a selection bias was introduced, which would likely overestimate adherence to the guidelines. Self‐reported responses may have introduced additional errors. Finally, the short interval between the release of the 2010 guidelines and the administration of the first survey may have contributed to the variability in implementation of some practices, but many of the recommendations had been previously included in the 2005 guidelines.

We conclude that there is wide variability between hospitals and within practices for resuscitation care. Future work should seek to understand which practices are associated with improved patient outcomes and how best to implement these practices in a more uniform fashion.

Acknowledgements

The authors thank Nancy Hinckley, who championed the study; David Chearo, Christelle Marpaud, and Martha Van Haitsma of the University of Chicago Survey Lab for their assistance in formulating and distributing the survey; and JoAnne Resnic, Nicole Twu, and Frank Zadravecz for administrative support.

Disclosures: This study was supported by the Society of Hospital Medicine with a grant from Philips Healthcare (Andover, MA). Dr. Edelson is supported by a career development award from the National Heart, Lung, and Blood Institute (K23 HL097157). In addition, she has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). Dr. Hunt has received research support from the Laerdal Foundation for Acute Medicine (Stavanger, Norway), the Hartwell Foundation (Memphis, TN), and the Arthur Vining Davis Foundation (Jacksonville, FL), and honoraria from the Kansas University Endowment (Kansas City, KS), JCCC (Overland Park, KS), and the UVA School of Medicine (Charlottesville, VA) and the European School of Management (Berlin, Germany). Dr. Mancini is supported in part by an Agency for Healthcare Research and Quality grant (R18HS020416). In addition, she has received research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and honoraria from Sotera Wireless, Inc. (San Diego, CA). Dr. Abella has received research support from the National Institutes of Health (NIH), Medtronic Foundation (Minneapolis, MN), and Philips Healthcare (Andover, MA); has volunteered with the American Heart Association; and received honoraria from Heartsine (Belfast, Ireland), Velomedix (Menlo Park, CA), and Stryker (Kalamazoo, MI). Mr. Miller is employed by the Society of Hospital Medicine.

Files
References
  1. Girotra S, Nallamothu BK, Spertus JA, Li Y, Krumholz HM, Chan PS. Trends in survival after in‐hospital cardiac arrest. N Engl J Med. 2012;367(20):19121920.
  2. Merchant RM, Yang L, Becker LB, et al. Incidence of treated cardiac arrest in hospitalized patients in the United States. Crit Care Med. 2011;39(11):24012406.
  3. Chan PS, Nichol G, Krumholz HM, et al. Racial differences in survival after in‐hospital cardiac arrest. JAMA. 2009;302(11):11951201.
  4. Chan PS, Nichol G, Krumholz HM, Spertus JA, Nallamothu BK. Hospital variation in time to defibrillation after in‐hospital cardiac arrest. Arch Intern Med. 2009;169(14):12651273.
  5. Goldberger ZD, Chan PS, Berg RA, et al. Duration of resuscitation efforts and survival after in‐hospital cardiac arrest: an observational study. Lancet. 2012;380(9852):14731481.
  6. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in‐hospital cardiac arrest. N Engl J Med. 2008;358(1):917.
  7. 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care science. Circulation. 2010;122(18 suppl 3):S640S946.
  8. Meaney PA, Bobrow BJ, Mancini ME, et al. Cardiopulmonary resuscitation quality: improving cardiac resuscitation outcomes both inside and outside the hospital: a consensus statement from the American Heart Association. Circulation. 2013;128(4):417435.
  9. American Hospital Association. 2008 AHA annual survey. AHA data viewer: survey instruments. 2012; Available at: http://www.ahadataviewer.com/about/hospital‐database. Accessed October 11, 2013.
  10. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 7th ed. Deerfield, IL: AAPOR; 2011.
  11. Lomas J, Anderson GM, Domnick‐Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 1989;321(19):13061311.
  12. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):14581465.
  13. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):12251230.
  14. Gabbott D, Smith G, Mitchell S, et al. Cardiopulmonary resuscitation standards for clinical practice and training in the UK. Accid Emerg Nurs. 2005;13(3):171179.
Article PDF
Issue
Journal of Hospital Medicine - 9(6)
Publications
Page Number
353-357
Sections
Files
Files
Article PDF
Article PDF

An estimated 200,000 adult patients suffer cardiac arrest in US hospitals each year, of which <20% survive to hospital discharge.[1, 2] Patient survival from in‐hospital cardiac arrest (IHCA), however, varies widely across hospitals, and may be partly attributed to differences in hospital practices.[3, 4, 5] Although there are data to support specific patient‐level practices in the hospital, such as delivery of electrical shock for ventricular fibrillation within 2 minutes of onset of the lethal rhythm,[6] little is known about in‐hospital systems‐level factors. Similar to patient‐level practices, some organizational and systems level practices are supported by international consensus and guideline recommendations.[7, 8] However, the adoption of these practices is poorly understood. As such, we sought to gain a better understanding of current US hospital practices with regard to IHCA and resuscitation with the hopes of identifying potential targets for improvement in quality and outcomes.

METHODS

We conducted a nationally representative mail survey between May 2011 and November 2011, targeting a stratified random sample of 1000 hospitals. We utilized the US Acute‐Care Hospitals (FY2008) database from the American Hospital Association to determine the total population of 3809 community hospitals (ie, nonfederal government, nonpsychiatric, and nonlong‐term care hospitals).[9] This included general medical and surgical, surgical, cancer, heart, orthopedic, and children's hospitals. These hospitals were stratified into tertiles by annual in‐patient days and teaching status (major, minor, nonteaching), from which our sample was randomly selected (Table 1). We identified each hospital's cardiopulmonary resuscitation (CPR) committee (sometimes known as code committee, code blue committee, or cardiac arrest committee) chair or chief medical/quality officer, to whom the paper‐based survey was addressed, with instructions to forward to the most appropriate person if someone other than the recipient. This study was evaluated by the University of Chicago institutional review board and deemed exempt from further review.

Figure 1
Hospital responders to in‐hospital resuscitations by institution type and level of participation. Bars represent the percent of hospitals reporting usual resuscitation responders in their hospitals, stratified by the teaching status of the hospital. Each bar is further subdivided by the likelihood of that provider to lead the resuscitation.

Survey

The survey content was developed by the study investigators and iteratively adapted by consensus and beta testing to require approximately 10 minutes to complete. Questions were edited and formatted by the University of Chicago Survey Lab (Chicago, IL) to be more precise and generalizable. Surveys were mailed in May 2011 and resent twice to nonresponders. A $10 incentive was included in the second mailing. When more than 1 response from a hospital was received, the more complete survey was used, or if equally complete, the responses were combined. All printing, mailing, receipt control, and data entry were performed by the University of Chicago Survey Lab, and data entry was double‐keyed to ensure accuracy.

Response rate was calculated based on the American Association for Public Opinion Research standard response rate formula.[10] It was assumed that the portion of nonresponding cases were ineligible at the same rate of cases for which eligibility was determined. A survey was considered complete if at least 75% of individual questions contained a valid response, partially complete if at least 40% but less than 75% of questions contained a valid response, and a nonresponse if less than 40% was completed. Nonresponses were excluded from the analysis.

Statistical Analysis

Analyses were performed using a statistical software application (Stata version 11.0; StataCorp, College Station, TX). Descriptive statistics were calculated and presented as number (%) or median (interquartile range). A [2] statistic was used to assess bias in response rate. We determined a priori 2 indicators of resource allocation (availability of a CPR committee and dedicated personnel for resuscitation quality improvement) and tested their association with quality improvement initiatives, using logistic regression to adjust for hospital teaching status and number of admissions as potential confounders. All tests of significance used a 2‐sided P<0.05.

RESULTS

Responses were received from 439 hospitals (425 complete and 14 partially complete), yielding a response rate of 44%. One subject ID was removed from the survey and could not be identified, so it was excluded from any analyses. Hospital demographics were similar between responders and nonresponders (P=0.50) (Table 1). Respondents who filled out the surveys included chief medical/quality officers (n=143 [33%]), chairs of CPR committees (n=64 [15%]), members of CPR committees (n=29 [7%]), chiefs of staff (n=33 [8%]), resuscitation officers/nurses (n=27 [6%]), chief nursing officers (n=13 [3%]), and others (n=131 [30%]).

Stratified Response Rates by Hospital Volume and Teaching Status
Teaching StatusAnnual Inpatient DaysTotal
<17,69517,695‐52,500>52,500
  • NOTE: Results are shown as number of respondents/total sampled (%).

Major1/2 (50)1/8 (13)40/82 (49)42/92 (46)
Minor13/39 (33)40/89 (45)62/133 (47)115/261 (44)
Nonteaching141/293 (48)100/236 (42)40/118 (34)281/647 (43)
Total156/335 (47)143/335 (43)145/336 (43)438/1,000 (44)

Table 2 summarizes structure, equipment, quality improvement, and pre‐ and postarrest practices across the hospitals. Of note, 77% of hospitals (n=334) reported having a predesignated, dedicated code team, and 66% (n=281) reported standardized defibrillator make and model throughout their hospital. However, less than one‐third of hospitals utilized any CPR assist technology (eg, CPR quality sensor or mechanical CPR device). The majority of hospitals reported having a rapid response team (RRT) (n=391 [91%]). Although a therapeutic hypothermia protocol for postarrest care was in place in over half of hospitals (n=252 [58%]), utilization of hypothermia for patients with return of spontaneous circulation was infrequent.

In‐hospital Resuscitation Structure and Practices
 Value2010 AHA Guidelines
  • NOTE: Results are shown as total (%) unless otherwise indicated. Percentages were adjusted by excluding missing responses. Abbreviations: AED, automatic external defibrillator; AHA, American Heart Association; CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; IQR, interquartile range; LOE, level of evidence; PA, public address; QI, quality improvement; ROSC, return of spontaneous circulation; RRT, rapid response team; TH, therapeutic hypothermia

  • These categories are not mutually exclusive

  • Recommended or supported in 2005 guidelines

  • May be considered for use in specific settings by properly trained personnel

  • Supported in the guidelines without official class recommendation.

Structure  
Existing CPR committee270 (66) 
CPR chair  
Physician only129 (48) 
Nurse only90 (34) 
Nurse/physician co‐chair31 (12) 
Other17 (6) 
Clinical specialty of chaira  
Pulmonary/critical care79 (35) 
Emergency medicine71 (31) 
Anesthesia/critical care43 (19) 
Cardiology38 (17) 
Other32 (14) 
Hospital medicine23 (10) 
Predetermined cardiac arrest team structure334 (77) 
Notifications of respondersa  
Hospital‐wide PA system406 (93) 
Pager/calls to individuals230 (53) 
Local alarm49 (11) 
Equipment  
AEDs used as primary defibrillator by location  
High‐acuity inpatient areas69 (16) 
Low‐acuity inpatient areas109 (26) 
Outpatient areas206 (51)Class IIb, LOE Cb
Public areas263 (78)Class IIb, LOE Cb
Defibrillator throughout hospital  
Same brand and model281 (66) 
Same brand, different models93 (22) 
Different brands54 (13) 
CPR assist technology useda  
None291 (70) 
Capnography106 (25)Class IIb, LOE Cb
Mechanical CPR25 (6)Class IIb, LOE B/Cbc
Feedback device17 (4)Class IIa, LOE B
Quality improvement  
IHCA tracked336 (82)Supportedbd
Data reviewed Supportedbd
Data not tracked/never reviewed85 (20) 
Intermittently53 (12) 
Routinely287 (68) 
Routine cardiac arrest case reviews/debriefing149 (34)Class IIa, LOE C
Dedicated staff to resuscitation QI196 (49) 
Full‐time equivalent staffing, median (IQR)0.5 (0.251.2) 
Routine simulated resuscitation training268 (62) 
Pre‐ and postarrest measures  
Hospitals with RRT391 (91)Class I, LOE Cb
Formal RRT‐specific training  
Never50 (14) 
Once110 (30) 
Recurrent163 (45) 
TH protocol/order set in place252 (58) 
Percent of patients with ROSC receiving TH Class IIb, LOE Bb
<5%309 (74) 
5%25%68 (16) 
26%50%11 (3) 
51%75%10 (2) 
>75%18 (4) 

Hospitals reported that routine responders to IHCA events included respiratory therapists (n=414 [95%]), critical care nurses (n=406 [93%]), floor nurses (n=396 [90%]), attending physicians (n=392 [89%]), physician trainees (n=162 [37%]), and pharmacists (n=210 [48%]). Figure 1 shows the distribution of responders and team leaders by hospital type. Of the nonteaching hospitals, attending‐level physicians were likely to respond at 94% (265/281) and routinely lead the resuscitations at 84% (236/281), whereas, of major teaching hospitals, attending physicians were only likely to respond at 71% (30/42) and routinely lead at 19% (8/42).

Two‐thirds of the hospitals had a CPR committee (n=270 [66%]), and 196 (49%) had some staff time dedicated to resuscitation quality improvement. Hospitals with a specific committee dedicated to resuscitation and/or dedicated staff for resuscitation quality improvement were more likely to routinely track cardiac arrest data (odds ratio [OR]: 3.64, 95% confidence interval [CI]: 2.056.47 and OR: 2.02, 95% CI: 1.16‐3.54, respectively) and review the data (OR: 2.67, 95% CI: 1.45‐4.92 and OR: 2.18, 95% CI: 1.22‐3.89, respectively), after adjusting for teaching status and hospital size. These hospitals were also more likely to engage in simulation training and debriefing (Table 3).

Correlation Between Resource Availability and Quality Improvement Practices
 CPR Committee, n=406Dedicated QI Staff, n=398
  • NOTE: Logistic regression adjusting for hospital size and teaching status was performed. All results are shown as odds ratio (95% confidence interval)

  • Abbreviations: CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; QI, quality improvement.

IHCA tracking3.64 (2.056.47)2.02 (1.16‐3.54)
Routinely review2.67 (1.45‐4.92)2.18 (1.22‐3.89)
Simulation training2.63 (1.66‐4.18)1.89 (1.24‐2.89)
Debriefing3.19 (1.89‐5.36)2.14 (1.39‐3.32)

Ninety percent (n=391) of respondents agreed that there is room for improvement in resuscitation practice at my hospital, and 70% (n=302) agreed that improved resuscitation would translate into improved patient outcomes. Overall, 78% (n=338) cited at least 1 barrier to improved resuscitation quality, of which the lack of adequate training (n=233 [54%]) and the lack of an appropriate champion (n=230 [53%]) were the most common. In subgroup analysis, nonteaching hospitals were significantly more likely to report the lack of a champion than their teaching counterparts (P=0.001) (Figure 2). In addition, we analyzed the data by hospitals that reported lack of a champion was not a barrier and compared them to those for whom it was, and found significantly higher adherence across all the measures in Table 2 supported by the 2010 guidelines, with the exception of real‐time feedback (data not shown).

Figure 2
Barriers to resuscitation quality improvement by institution type. Bars represent the percent of responders reporting specific perceived barriers to resuscitation quality improvement at their hospital, stratified by the teaching status of the hospital.

DISCUSSION

In this nationally representative sample of hospitals, we found considerable variability in cardiac arrest and resuscitation structures and processes, suggesting potential areas to target for improvement. Some practices, including use of RRTs and defibrillator standardization, were fairly routine, whereas others, such as therapeutic hypothermia and CPR assist technology, were rarely utilized. Quality initiatives, such as data tracking and review, simulation training, and debriefing were variable.

Several factors likely contribute to the variable implementation of evidence‐based practices. Guidelines alone have been shown to have little impact on practice by physicians in general.[11] This is supported by the lack of correlation we found between the presence, absence or strength of specific American Heart Association (AHA) emergency cardiovascular care treatment recommendations and the percent of hospitals reporting performing that measure. It is possible that other factors, such as a lack of familiarity or agreement with those guidelines, or the presence of external barriers, may be contributing.[12, 13] Specifically, the importance of a clinical champion was supported by our finding that hospitals reporting lack of a champion as a barrier were less likely to be adherent with guidelines. However, because the study did not directly test the impact of a champion, we wanted to be careful to avoid overstating or editorializing our results.

Some of the variability may also be related to the resource intensiveness of the practice. Routine simulation training and debriefing interventions, for example, are time intensive and require trained personnel to institute. That may explain the correlation we noted between these practices and the presence of CPR committee and dedicated personnel. The use of dedicated personnel was rare in this study, with less than half of respondents reporting any dedicated staff and a median of 0.5 full‐time equivalents for those reporting positively. This is in stark contrast to the routine use of resuscitation officers (primarily nurses dedicated to overseeing resuscitation practices and education at the hospital) in the United Kingdom.[14] Such a resuscitation officer model adopted by US hospitals could improve the quality and intensity of resuscitation care approaches.

Particularly surprising was the high rate of respondents (70%) reporting that they do not utilize any CPR assist technology. In the patient who does not have an arterial line, use of quantitative capnography is the best measure of cardiac output during cardiac arrest, yet only one‐quarter of hospitals reported using it, with no discrepancy between hospital type or size. A recent summit of national resuscitation experts expounded on the AHA guidelines suggesting that end‐tidal carbon dioxide should be used in all arrests to guide the quality of CPR with a goal value of >20.[8] Similarly, CPR feedback devices have an even higher level of evidence recommendation in the 2010 AHA guidelines than capnography, yet only 4% of hospitals reported utilizing them. Although it is true that introducing these CPR assist technologies into a hospital would require some effort on the part of hospital leadership, it is important to recognize the potential role such devices might play in the larger context of a resuscitation quality program to optimize clinical outcomes from IHCA.

Several differences were noted between hospitals based on teaching status. Although all hospitals were more likely to rely on physicians to lead resuscitations, nonteaching hospitals were more likely to report routine leadership by nurses and pharmacists. Nonteaching hospitals were also less likely to have a CPR committee, even after adjusting for hospital size. In addition, these hospitals were also more likely to report the lack of a clinical champion as a barrier to quality improvement.

There were several limitations to this study. First, this was a descriptive survey that was not tied to outcomes. As such, we are unable to draw conclusions about which practices correlate with decreased incidence of cardiac arrest and improved survival. Second, this was an optional survey with a somewhat limited response rate. Even though the characteristics of the nonresponding hospitals were similar to the responding hospitals, we cannot rule out the possibility that a selection bias was introduced, which would likely overestimate adherence to the guidelines. Self‐reported responses may have introduced additional errors. Finally, the short interval between the release of the 2010 guidelines and the administration of the first survey may have contributed to the variability in implementation of some practices, but many of the recommendations had been previously included in the 2005 guidelines.

We conclude that there is wide variability between hospitals and within practices for resuscitation care. Future work should seek to understand which practices are associated with improved patient outcomes and how best to implement these practices in a more uniform fashion.

Acknowledgements

The authors thank Nancy Hinckley, who championed the study; David Chearo, Christelle Marpaud, and Martha Van Haitsma of the University of Chicago Survey Lab for their assistance in formulating and distributing the survey; and JoAnne Resnic, Nicole Twu, and Frank Zadravecz for administrative support.

Disclosures: This study was supported by the Society of Hospital Medicine with a grant from Philips Healthcare (Andover, MA). Dr. Edelson is supported by a career development award from the National Heart, Lung, and Blood Institute (K23 HL097157). In addition, she has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). Dr. Hunt has received research support from the Laerdal Foundation for Acute Medicine (Stavanger, Norway), the Hartwell Foundation (Memphis, TN), and the Arthur Vining Davis Foundation (Jacksonville, FL), and honoraria from the Kansas University Endowment (Kansas City, KS), JCCC (Overland Park, KS), and the UVA School of Medicine (Charlottesville, VA) and the European School of Management (Berlin, Germany). Dr. Mancini is supported in part by an Agency for Healthcare Research and Quality grant (R18HS020416). In addition, she has received research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and honoraria from Sotera Wireless, Inc. (San Diego, CA). Dr. Abella has received research support from the National Institutes of Health (NIH), Medtronic Foundation (Minneapolis, MN), and Philips Healthcare (Andover, MA); has volunteered with the American Heart Association; and received honoraria from Heartsine (Belfast, Ireland), Velomedix (Menlo Park, CA), and Stryker (Kalamazoo, MI). Mr. Miller is employed by the Society of Hospital Medicine.

An estimated 200,000 adult patients suffer cardiac arrest in US hospitals each year, of which <20% survive to hospital discharge.[1, 2] Patient survival from in‐hospital cardiac arrest (IHCA), however, varies widely across hospitals, and may be partly attributed to differences in hospital practices.[3, 4, 5] Although there are data to support specific patient‐level practices in the hospital, such as delivery of electrical shock for ventricular fibrillation within 2 minutes of onset of the lethal rhythm,[6] little is known about in‐hospital systems‐level factors. Similar to patient‐level practices, some organizational and systems level practices are supported by international consensus and guideline recommendations.[7, 8] However, the adoption of these practices is poorly understood. As such, we sought to gain a better understanding of current US hospital practices with regard to IHCA and resuscitation with the hopes of identifying potential targets for improvement in quality and outcomes.

METHODS

We conducted a nationally representative mail survey between May 2011 and November 2011, targeting a stratified random sample of 1000 hospitals. We utilized the US Acute‐Care Hospitals (FY2008) database from the American Hospital Association to determine the total population of 3809 community hospitals (ie, nonfederal government, nonpsychiatric, and nonlong‐term care hospitals).[9] This included general medical and surgical, surgical, cancer, heart, orthopedic, and children's hospitals. These hospitals were stratified into tertiles by annual in‐patient days and teaching status (major, minor, nonteaching), from which our sample was randomly selected (Table 1). We identified each hospital's cardiopulmonary resuscitation (CPR) committee (sometimes known as code committee, code blue committee, or cardiac arrest committee) chair or chief medical/quality officer, to whom the paper‐based survey was addressed, with instructions to forward to the most appropriate person if someone other than the recipient. This study was evaluated by the University of Chicago institutional review board and deemed exempt from further review.

Figure 1
Hospital responders to in‐hospital resuscitations by institution type and level of participation. Bars represent the percent of hospitals reporting usual resuscitation responders in their hospitals, stratified by the teaching status of the hospital. Each bar is further subdivided by the likelihood of that provider to lead the resuscitation.

Survey

The survey content was developed by the study investigators and iteratively adapted by consensus and beta testing to require approximately 10 minutes to complete. Questions were edited and formatted by the University of Chicago Survey Lab (Chicago, IL) to be more precise and generalizable. Surveys were mailed in May 2011 and resent twice to nonresponders. A $10 incentive was included in the second mailing. When more than 1 response from a hospital was received, the more complete survey was used, or if equally complete, the responses were combined. All printing, mailing, receipt control, and data entry were performed by the University of Chicago Survey Lab, and data entry was double‐keyed to ensure accuracy.

Response rate was calculated based on the American Association for Public Opinion Research standard response rate formula.[10] It was assumed that the portion of nonresponding cases were ineligible at the same rate of cases for which eligibility was determined. A survey was considered complete if at least 75% of individual questions contained a valid response, partially complete if at least 40% but less than 75% of questions contained a valid response, and a nonresponse if less than 40% was completed. Nonresponses were excluded from the analysis.

Statistical Analysis

Analyses were performed using a statistical software application (Stata version 11.0; StataCorp, College Station, TX). Descriptive statistics were calculated and presented as number (%) or median (interquartile range). A [2] statistic was used to assess bias in response rate. We determined a priori 2 indicators of resource allocation (availability of a CPR committee and dedicated personnel for resuscitation quality improvement) and tested their association with quality improvement initiatives, using logistic regression to adjust for hospital teaching status and number of admissions as potential confounders. All tests of significance used a 2‐sided P<0.05.

RESULTS

Responses were received from 439 hospitals (425 complete and 14 partially complete), yielding a response rate of 44%. One subject ID was removed from the survey and could not be identified, so it was excluded from any analyses. Hospital demographics were similar between responders and nonresponders (P=0.50) (Table 1). Respondents who filled out the surveys included chief medical/quality officers (n=143 [33%]), chairs of CPR committees (n=64 [15%]), members of CPR committees (n=29 [7%]), chiefs of staff (n=33 [8%]), resuscitation officers/nurses (n=27 [6%]), chief nursing officers (n=13 [3%]), and others (n=131 [30%]).

Stratified Response Rates by Hospital Volume and Teaching Status
Teaching StatusAnnual Inpatient DaysTotal
<17,69517,695‐52,500>52,500
  • NOTE: Results are shown as number of respondents/total sampled (%).

Major1/2 (50)1/8 (13)40/82 (49)42/92 (46)
Minor13/39 (33)40/89 (45)62/133 (47)115/261 (44)
Nonteaching141/293 (48)100/236 (42)40/118 (34)281/647 (43)
Total156/335 (47)143/335 (43)145/336 (43)438/1,000 (44)

Table 2 summarizes structure, equipment, quality improvement, and pre‐ and postarrest practices across the hospitals. Of note, 77% of hospitals (n=334) reported having a predesignated, dedicated code team, and 66% (n=281) reported standardized defibrillator make and model throughout their hospital. However, less than one‐third of hospitals utilized any CPR assist technology (eg, CPR quality sensor or mechanical CPR device). The majority of hospitals reported having a rapid response team (RRT) (n=391 [91%]). Although a therapeutic hypothermia protocol for postarrest care was in place in over half of hospitals (n=252 [58%]), utilization of hypothermia for patients with return of spontaneous circulation was infrequent.

In‐hospital Resuscitation Structure and Practices
 Value2010 AHA Guidelines
  • NOTE: Results are shown as total (%) unless otherwise indicated. Percentages were adjusted by excluding missing responses. Abbreviations: AED, automatic external defibrillator; AHA, American Heart Association; CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; IQR, interquartile range; LOE, level of evidence; PA, public address; QI, quality improvement; ROSC, return of spontaneous circulation; RRT, rapid response team; TH, therapeutic hypothermia

  • These categories are not mutually exclusive

  • Recommended or supported in 2005 guidelines

  • May be considered for use in specific settings by properly trained personnel

  • Supported in the guidelines without official class recommendation.

Structure  
Existing CPR committee270 (66) 
CPR chair  
Physician only129 (48) 
Nurse only90 (34) 
Nurse/physician co‐chair31 (12) 
Other17 (6) 
Clinical specialty of chaira  
Pulmonary/critical care79 (35) 
Emergency medicine71 (31) 
Anesthesia/critical care43 (19) 
Cardiology38 (17) 
Other32 (14) 
Hospital medicine23 (10) 
Predetermined cardiac arrest team structure334 (77) 
Notifications of respondersa  
Hospital‐wide PA system406 (93) 
Pager/calls to individuals230 (53) 
Local alarm49 (11) 
Equipment  
AEDs used as primary defibrillator by location  
High‐acuity inpatient areas69 (16) 
Low‐acuity inpatient areas109 (26) 
Outpatient areas206 (51)Class IIb, LOE Cb
Public areas263 (78)Class IIb, LOE Cb
Defibrillator throughout hospital  
Same brand and model281 (66) 
Same brand, different models93 (22) 
Different brands54 (13) 
CPR assist technology useda  
None291 (70) 
Capnography106 (25)Class IIb, LOE Cb
Mechanical CPR25 (6)Class IIb, LOE B/Cbc
Feedback device17 (4)Class IIa, LOE B
Quality improvement  
IHCA tracked336 (82)Supportedbd
Data reviewed Supportedbd
Data not tracked/never reviewed85 (20) 
Intermittently53 (12) 
Routinely287 (68) 
Routine cardiac arrest case reviews/debriefing149 (34)Class IIa, LOE C
Dedicated staff to resuscitation QI196 (49) 
Full‐time equivalent staffing, median (IQR)0.5 (0.251.2) 
Routine simulated resuscitation training268 (62) 
Pre‐ and postarrest measures  
Hospitals with RRT391 (91)Class I, LOE Cb
Formal RRT‐specific training  
Never50 (14) 
Once110 (30) 
Recurrent163 (45) 
TH protocol/order set in place252 (58) 
Percent of patients with ROSC receiving TH Class IIb, LOE Bb
<5%309 (74) 
5%25%68 (16) 
26%50%11 (3) 
51%75%10 (2) 
>75%18 (4) 

Hospitals reported that routine responders to IHCA events included respiratory therapists (n=414 [95%]), critical care nurses (n=406 [93%]), floor nurses (n=396 [90%]), attending physicians (n=392 [89%]), physician trainees (n=162 [37%]), and pharmacists (n=210 [48%]). Figure 1 shows the distribution of responders and team leaders by hospital type. Of the nonteaching hospitals, attending‐level physicians were likely to respond at 94% (265/281) and routinely lead the resuscitations at 84% (236/281), whereas, of major teaching hospitals, attending physicians were only likely to respond at 71% (30/42) and routinely lead at 19% (8/42).

Two‐thirds of the hospitals had a CPR committee (n=270 [66%]), and 196 (49%) had some staff time dedicated to resuscitation quality improvement. Hospitals with a specific committee dedicated to resuscitation and/or dedicated staff for resuscitation quality improvement were more likely to routinely track cardiac arrest data (odds ratio [OR]: 3.64, 95% confidence interval [CI]: 2.056.47 and OR: 2.02, 95% CI: 1.16‐3.54, respectively) and review the data (OR: 2.67, 95% CI: 1.45‐4.92 and OR: 2.18, 95% CI: 1.22‐3.89, respectively), after adjusting for teaching status and hospital size. These hospitals were also more likely to engage in simulation training and debriefing (Table 3).

Correlation Between Resource Availability and Quality Improvement Practices
 CPR Committee, n=406Dedicated QI Staff, n=398
  • NOTE: Logistic regression adjusting for hospital size and teaching status was performed. All results are shown as odds ratio (95% confidence interval)

  • Abbreviations: CPR, cardiopulmonary resuscitation; IHCA, in‐hospital cardiac arrest; QI, quality improvement.

IHCA tracking3.64 (2.056.47)2.02 (1.16‐3.54)
Routinely review2.67 (1.45‐4.92)2.18 (1.22‐3.89)
Simulation training2.63 (1.66‐4.18)1.89 (1.24‐2.89)
Debriefing3.19 (1.89‐5.36)2.14 (1.39‐3.32)

Ninety percent (n=391) of respondents agreed that there is room for improvement in resuscitation practice at my hospital, and 70% (n=302) agreed that improved resuscitation would translate into improved patient outcomes. Overall, 78% (n=338) cited at least 1 barrier to improved resuscitation quality, of which the lack of adequate training (n=233 [54%]) and the lack of an appropriate champion (n=230 [53%]) were the most common. In subgroup analysis, nonteaching hospitals were significantly more likely to report the lack of a champion than their teaching counterparts (P=0.001) (Figure 2). In addition, we analyzed the data by hospitals that reported lack of a champion was not a barrier and compared them to those for whom it was, and found significantly higher adherence across all the measures in Table 2 supported by the 2010 guidelines, with the exception of real‐time feedback (data not shown).

Figure 2
Barriers to resuscitation quality improvement by institution type. Bars represent the percent of responders reporting specific perceived barriers to resuscitation quality improvement at their hospital, stratified by the teaching status of the hospital.

DISCUSSION

In this nationally representative sample of hospitals, we found considerable variability in cardiac arrest and resuscitation structures and processes, suggesting potential areas to target for improvement. Some practices, including use of RRTs and defibrillator standardization, were fairly routine, whereas others, such as therapeutic hypothermia and CPR assist technology, were rarely utilized. Quality initiatives, such as data tracking and review, simulation training, and debriefing were variable.

Several factors likely contribute to the variable implementation of evidence‐based practices. Guidelines alone have been shown to have little impact on practice by physicians in general.[11] This is supported by the lack of correlation we found between the presence, absence or strength of specific American Heart Association (AHA) emergency cardiovascular care treatment recommendations and the percent of hospitals reporting performing that measure. It is possible that other factors, such as a lack of familiarity or agreement with those guidelines, or the presence of external barriers, may be contributing.[12, 13] Specifically, the importance of a clinical champion was supported by our finding that hospitals reporting lack of a champion as a barrier were less likely to be adherent with guidelines. However, because the study did not directly test the impact of a champion, we wanted to be careful to avoid overstating or editorializing our results.

Some of the variability may also be related to the resource intensiveness of the practice. Routine simulation training and debriefing interventions, for example, are time intensive and require trained personnel to institute. That may explain the correlation we noted between these practices and the presence of CPR committee and dedicated personnel. The use of dedicated personnel was rare in this study, with less than half of respondents reporting any dedicated staff and a median of 0.5 full‐time equivalents for those reporting positively. This is in stark contrast to the routine use of resuscitation officers (primarily nurses dedicated to overseeing resuscitation practices and education at the hospital) in the United Kingdom.[14] Such a resuscitation officer model adopted by US hospitals could improve the quality and intensity of resuscitation care approaches.

Particularly surprising was the high rate of respondents (70%) reporting that they do not utilize any CPR assist technology. In the patient who does not have an arterial line, use of quantitative capnography is the best measure of cardiac output during cardiac arrest, yet only one‐quarter of hospitals reported using it, with no discrepancy between hospital type or size. A recent summit of national resuscitation experts expounded on the AHA guidelines suggesting that end‐tidal carbon dioxide should be used in all arrests to guide the quality of CPR with a goal value of >20.[8] Similarly, CPR feedback devices have an even higher level of evidence recommendation in the 2010 AHA guidelines than capnography, yet only 4% of hospitals reported utilizing them. Although it is true that introducing these CPR assist technologies into a hospital would require some effort on the part of hospital leadership, it is important to recognize the potential role such devices might play in the larger context of a resuscitation quality program to optimize clinical outcomes from IHCA.

Several differences were noted between hospitals based on teaching status. Although all hospitals were more likely to rely on physicians to lead resuscitations, nonteaching hospitals were more likely to report routine leadership by nurses and pharmacists. Nonteaching hospitals were also less likely to have a CPR committee, even after adjusting for hospital size. In addition, these hospitals were also more likely to report the lack of a clinical champion as a barrier to quality improvement.

There were several limitations to this study. First, this was a descriptive survey that was not tied to outcomes. As such, we are unable to draw conclusions about which practices correlate with decreased incidence of cardiac arrest and improved survival. Second, this was an optional survey with a somewhat limited response rate. Even though the characteristics of the nonresponding hospitals were similar to the responding hospitals, we cannot rule out the possibility that a selection bias was introduced, which would likely overestimate adherence to the guidelines. Self‐reported responses may have introduced additional errors. Finally, the short interval between the release of the 2010 guidelines and the administration of the first survey may have contributed to the variability in implementation of some practices, but many of the recommendations had been previously included in the 2005 guidelines.

We conclude that there is wide variability between hospitals and within practices for resuscitation care. Future work should seek to understand which practices are associated with improved patient outcomes and how best to implement these practices in a more uniform fashion.

Acknowledgements

The authors thank Nancy Hinckley, who championed the study; David Chearo, Christelle Marpaud, and Martha Van Haitsma of the University of Chicago Survey Lab for their assistance in formulating and distributing the survey; and JoAnne Resnic, Nicole Twu, and Frank Zadravecz for administrative support.

Disclosures: This study was supported by the Society of Hospital Medicine with a grant from Philips Healthcare (Andover, MA). Dr. Edelson is supported by a career development award from the National Heart, Lung, and Blood Institute (K23 HL097157). In addition, she has received research support and honoraria from Philips Healthcare (Andover, MA), research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and an honorarium from Early Sense (Tel Aviv, Israel). Dr. Hunt has received research support from the Laerdal Foundation for Acute Medicine (Stavanger, Norway), the Hartwell Foundation (Memphis, TN), and the Arthur Vining Davis Foundation (Jacksonville, FL), and honoraria from the Kansas University Endowment (Kansas City, KS), JCCC (Overland Park, KS), and the UVA School of Medicine (Charlottesville, VA) and the European School of Management (Berlin, Germany). Dr. Mancini is supported in part by an Agency for Healthcare Research and Quality grant (R18HS020416). In addition, she has received research support from the American Heart Association (Dallas, TX) and Laerdal Medical (Stavanger, Norway), and honoraria from Sotera Wireless, Inc. (San Diego, CA). Dr. Abella has received research support from the National Institutes of Health (NIH), Medtronic Foundation (Minneapolis, MN), and Philips Healthcare (Andover, MA); has volunteered with the American Heart Association; and received honoraria from Heartsine (Belfast, Ireland), Velomedix (Menlo Park, CA), and Stryker (Kalamazoo, MI). Mr. Miller is employed by the Society of Hospital Medicine.

References
  1. Girotra S, Nallamothu BK, Spertus JA, Li Y, Krumholz HM, Chan PS. Trends in survival after in‐hospital cardiac arrest. N Engl J Med. 2012;367(20):19121920.
  2. Merchant RM, Yang L, Becker LB, et al. Incidence of treated cardiac arrest in hospitalized patients in the United States. Crit Care Med. 2011;39(11):24012406.
  3. Chan PS, Nichol G, Krumholz HM, et al. Racial differences in survival after in‐hospital cardiac arrest. JAMA. 2009;302(11):11951201.
  4. Chan PS, Nichol G, Krumholz HM, Spertus JA, Nallamothu BK. Hospital variation in time to defibrillation after in‐hospital cardiac arrest. Arch Intern Med. 2009;169(14):12651273.
  5. Goldberger ZD, Chan PS, Berg RA, et al. Duration of resuscitation efforts and survival after in‐hospital cardiac arrest: an observational study. Lancet. 2012;380(9852):14731481.
  6. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in‐hospital cardiac arrest. N Engl J Med. 2008;358(1):917.
  7. 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care science. Circulation. 2010;122(18 suppl 3):S640S946.
  8. Meaney PA, Bobrow BJ, Mancini ME, et al. Cardiopulmonary resuscitation quality: improving cardiac resuscitation outcomes both inside and outside the hospital: a consensus statement from the American Heart Association. Circulation. 2013;128(4):417435.
  9. American Hospital Association. 2008 AHA annual survey. AHA data viewer: survey instruments. 2012; Available at: http://www.ahadataviewer.com/about/hospital‐database. Accessed October 11, 2013.
  10. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 7th ed. Deerfield, IL: AAPOR; 2011.
  11. Lomas J, Anderson GM, Domnick‐Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 1989;321(19):13061311.
  12. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):14581465.
  13. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):12251230.
  14. Gabbott D, Smith G, Mitchell S, et al. Cardiopulmonary resuscitation standards for clinical practice and training in the UK. Accid Emerg Nurs. 2005;13(3):171179.
References
  1. Girotra S, Nallamothu BK, Spertus JA, Li Y, Krumholz HM, Chan PS. Trends in survival after in‐hospital cardiac arrest. N Engl J Med. 2012;367(20):19121920.
  2. Merchant RM, Yang L, Becker LB, et al. Incidence of treated cardiac arrest in hospitalized patients in the United States. Crit Care Med. 2011;39(11):24012406.
  3. Chan PS, Nichol G, Krumholz HM, et al. Racial differences in survival after in‐hospital cardiac arrest. JAMA. 2009;302(11):11951201.
  4. Chan PS, Nichol G, Krumholz HM, Spertus JA, Nallamothu BK. Hospital variation in time to defibrillation after in‐hospital cardiac arrest. Arch Intern Med. 2009;169(14):12651273.
  5. Goldberger ZD, Chan PS, Berg RA, et al. Duration of resuscitation efforts and survival after in‐hospital cardiac arrest: an observational study. Lancet. 2012;380(9852):14731481.
  6. Chan PS, Krumholz HM, Nichol G, Nallamothu BK. Delayed time to defibrillation after in‐hospital cardiac arrest. N Engl J Med. 2008;358(1):917.
  7. 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care science. Circulation. 2010;122(18 suppl 3):S640S946.
  8. Meaney PA, Bobrow BJ, Mancini ME, et al. Cardiopulmonary resuscitation quality: improving cardiac resuscitation outcomes both inside and outside the hospital: a consensus statement from the American Heart Association. Circulation. 2013;128(4):417435.
  9. American Hospital Association. 2008 AHA annual survey. AHA data viewer: survey instruments. 2012; Available at: http://www.ahadataviewer.com/about/hospital‐database. Accessed October 11, 2013.
  10. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 7th ed. Deerfield, IL: AAPOR; 2011.
  11. Lomas J, Anderson GM, Domnick‐Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med. 1989;321(19):13061311.
  12. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):14581465.
  13. Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):12251230.
  14. Gabbott D, Smith G, Mitchell S, et al. Cardiopulmonary resuscitation standards for clinical practice and training in the UK. Accid Emerg Nurs. 2005;13(3):171179.
Issue
Journal of Hospital Medicine - 9(6)
Issue
Journal of Hospital Medicine - 9(6)
Page Number
353-357
Page Number
353-357
Publications
Publications
Article Type
Display Headline
Hospital cardiac arrest resuscitation practice in the United States: A nationally representative survey
Display Headline
Hospital cardiac arrest resuscitation practice in the United States: A nationally representative survey
Sections
Article Source

© 2014 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, Section of Hospital Medicine, University of Chicago, 5841 S. Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773‐834‐2191; Fax: 773‐795‐7398; E‐mail: dperes@uchicago.edu
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files

Patient Acuity Rating

Article Type
Changed
Wed, 05/23/2018 - 14:44
Display Headline
Patient acuity rating: Quantifying clinical judgment regarding inpatient stability

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1
Patient Acuity Rating scale. Abbreviations: ICU, intensive care unit.

Study Setting and Participants

This study was conducted at The University of Chicago Medical Center, an academic, tertiary care facility with approximately 600 inpatient beds. Subjects involved both the clinicians who provided PAR scores and the patients upon whom the PAR scores and outcomes were based. The clinicians included internal medicine interns, residents, and attending physicians, as well as midlevel providers (nurse practitioners or physician assistants). Clinicians were eligible for inclusion if they cared for patients on one of nine adult ward services between January and June 2008. They were included in the study if they consented to participate. Housestaff, with medicine attending supervision, covered patients on seven general medicine services, while midlevel practitioners, also with medicine attending supervision, covered patients on two hepatology and transplant services.

Providers were independently surveyed once per call‐cycle (every 2 to 4 days depending on the service) by study personnel regarding each of their patients, and instructed not to consult with other members of the team regarding their PAR score assignments. All patients for whom a participating clinician provided a PAR score were included in the analysis. Clinician subjects were carefully surveyed at the end of their work day, just prior to, or immediately following, their handover to the cross‐covering physician, so as to minimize the risk that they might alter their plan and transfer a patient to the ICU based on the PAR score.

Data Analysis

PAR scores were entered into a database (Excel, Microsoft Corporation, Redmond, WA) and then linked to patient demographic and outcome data obtained from hospital administrative databases. Weighted kappa statistics were used to evaluate inter‐rater reliability. Ordinal trend testing was used to correlate the PAR with patient outcomes by provider. In addition, receiver operator characteristics (ROC) curves were constructed, and area under the curve (AUC) calculated and compared among providers using paired chi‐squared statistics. Sensitivities and specificities were determined for each theoretical PAR cutoff. Clustered multivariate logistic regression was used to adjust for provider, service, and individual patient. All calculations were performed using a statistical software application (Stata, College Station, TX).

Approval

The study protocol, consent, and data collection mechanisms were approved by the Institutional Review Board of the University of Chicago Medical Center. Waiver of consent provisions were used for patients on the basis of minimal harm and general impracticability, while a written consent process was used for patient care providers. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 regulations.

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

Patient and Admission Characteristics
CharacteristicGeneral Medicine Teaching ServicesMultispecialty Non‐Teaching ServicesP‐Value
  • Abbreviation: N/A, not applicable.

Patients (n)1,373290NA
Admissions (n)1,660359NA
Age, mean (SD) years57 (21)57 (13)0.73
Women, n (%)1,006 (61)173 (48)<0.001
Race, n (%)  <0.001
White203 (12)133 (37) 
Black1,129 (68)125 (35) 
Hispanic26 (2)34 (9) 
Asian11 (1)10 (3) 
Other/unknown291 (18)57 (16) 
Severity of illness, n (%)  <0.001
Minor121 (7)2 (1) 
Moderate461 (28)44 (12) 
Major677 (41)179 (50) 
Extreme329 (20)123 (34) 
N/A77 (4)11 (3) 
Discharged home, n (%)1,347 (81)282 (79)0.25
Expired (not hospice), n (%)25 (2)28 (8)<0.001

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Weighted Kappa Statistics by Provider Pair
Provider PairObservations (n)Agreement (%)Weighted Kappa
Interns vs residents1,00687.10.42
Residents vs attendings1,01282.50.35
Interns vs attendings1,02684.40.32
Midlevels vs attendings20885.00.43

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Patient Acuity Rating (PAR) Sensitivities and Specificities
PARAll Patients (n)Decompensating Patients (n)Sensitivity (%)Specificity (%)
7401216.299.2
61843040.595.4
55614662.284.6
41,1206182.468.3
32,0446993.241.0
23,0057398.612.3
13,41974100.00.0
Area Under the Patient Acuity Rating (PAR) Receiver Operator Characteristics Curve by Provider
ServiceProviderObservations (n)PAR, median (IQR)AUROC (95% CI)
  • Abbreviations: AUROC, area under the receiver operator characteristics curve; CI, confidence interval; IQR, interquartile range.

  • The only significant difference in pair‐wise comparison occurred between residents and attendings (P = 0.01).

General medicineInterns1,5673 (2‐4)0.79 (0.70, 0.88)
General medicineResidents1,6113 (2‐4)0.69 (0.59, 0.78)*
General medicineAttendings1,7913 (2‐4)0.84 (0.78, 0.90)*
MultispecialtyAttendings8233 (2‐4)0.88 (0.79, 0.97)
MultispecialtyMidlevels2423 (2‐4)0.80 (0.64, 0.95)
CombinedAll3,4193 (2‐4)0.82 (0.77, 0.87)
Figure 2
Percent of patients showing clinical deterioration to the point of cardiac arrest or intensive care unit (ICU) transfer by Patient Acuity Rating (PAR) and provider. Ordinal trend P‐value was <0.001 for each provider.

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Acknowledgements

The authors are grateful for the support and dedication of the residents, faculty, and staff in the Department of Medicine at the University of Chicago, without whom this study would not have been possible. They also thank Trevor C. Yuen for statistical support, David Beiser, MD, MS, and Kenneth Rasinski, PhD, for scientific advice, Kate Weaver for expert administrative support and Deborah Walsh, RN, MS, Jefferson Cua, and Amanda Schmitz for assistance with data collection.

Files
References
  1. Accreditation Council for Graduate Medical Education (ACGME).Task Force on Quality Care and Professionalism.Proposed standards: common program requirements.2007. Available at: http://acgme‐2010standards.org/pdf/Proposed_Standards.pdf. Accessed August 16,year="2010"2010.
  2. Arora VM,Georgitis E,Siddique J, et al.Association of workload of on‐call medical interns with on‐call sleep duration, shift duration, and participation in educational activities.JAMA.2008;300(10):11461153.
  3. Institute of Medicine (IOM).Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety.Report brief on resident duty hours: enhancing sleep, supervision, and safety.Washington, DC:National Academy Press;2008. Available at: http://www.iom.edu/∼/media/Files/Report%20Files/2008/Resident‐Duty‐Hours/residency%20hours%20revised% 20for%20web.pdf. Accessed August 16,year="2010"2010.
  4. Kuo PC,Douglas AR,Oleski D,Jacobs DO,Schroeder RA.Determining benchmarks for evaluation and management coding in an academic division of general surgery.J Am Coll Surg.2004;199(1):124130.
  5. Shen YJ.Applying the 3M all patient refined diagnosis related groups grouper to measure inpatient severity in the VA.Med Care.2003;41(6):103110.
  6. Norman G,Young M,Brooks L.Non‐analytical models of clinical reasoning: the role of experience.Med Educ.2007;41(12):11401145.
  7. Arora V,Johnson J,Lovinger D,Humphrey HJ,Meltzer DO.Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis.Qual Saf Health Care.2005;14(6):401407.
  8. Gandhi TK.Fumbled handoffs: one dropped ball after another.Ann Intern Med.2005;142(5):352358.
  9. Horwitz LI,Moin T,Krumholz HM,Wang L,Bradley EH.What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff.Qual Saf Health Care.2009;18(4):248255.
  10. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  11. Knaus WA,Harrell FE,Lynn J, et al.The SUPPORT prognostic model—objective estimates of survival for seriously ill hospitalized adults.Ann Intern Med.1995;122(3):191203.
  12. Meadow W,Frain L,Ren YY,Lee G,Soneji S,Lantos J.Serial assessment of mortality in the neonatal intensive care unit by algorithm and intuition: certainty, uncertainty, and informed consent.Pediatrics.2002;109(5):878886.
  13. Sinuff T,Adhikari NKJ,Cook DJ, et al.Mortality predictions in the intensive care unit: comparing physicians with scoring systems.Crit Care Med.2006;34(3):878885.
  14. Smith GB,Prytherch DR,Schmidt PL,Featherstone PI.Review and performance evaluation of aggregate weighted ‘track and trigger’ systems.Resuscitation.2008;77(2):170179.
  15. Kho A,Rotz D,Alrahi K, et al.Utility of commonly captured data from an EHR to identify hospitalized patients at risk for clinical deterioration.AMIA Annu Symp Proc.2007:404408.
  16. Jha AK,DesRoches CM,Campbell EG, et al.Use of electronic health records in U.S. hospitals.N Engl J Med.2009;360(16):16281638.
  17. Lovett PB,Buchwald JM,Sturmann K,Bijur P.The vexatious vital: neither clinical measurements by nurses nor an electronic monitor provides accurate measurements of respiratory rate in triage.Ann Emerg Med.2005;45(1):6876.
  18. Laurila JV,Pitkala KH,Strandberg TE,Tilvis RS.Detection and documentation of dementia and delirium in acute geriatric wards.Gen Hosp Psychiatry.2004;26(1):3135.
  19. Hillman K,Parr M,Flabouris A,Bishop G,Stewart A.Redefining in‐hospital resuscitation: the concept of the medical emergency team.Resuscitation.2001;48(2):105110.
  20. Hodgetts TJ,Kenward G,Vlachonikolis IG,Payne S,Castle N.The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team.Resuscitation.2002;54(2):125131.
  21. Brilli RJ,Gibson R,Luria JW, et al.Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit.Pediatr Crit Care Med.2007;8(3):236246.
  22. Tibballs J,Kinney S,Duke T,Oakley E,Hennessy M.Reduction of paediatric in‐patient cardiac arrest and death with a medical emergency team: preliminary results.Arch Dis Child.2005;90(11):11481152.
  23. Guarisco S,Oddone E,Simel D.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9(5):272277.
  24. Nerenz D,Rosman H,Newcomb C, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150(11):22942297.
  25. Dresselhaus TR,Luck J,Wright BC,Spragg RG,Lee ML,Bozzette SA.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13(8):534540.
Article PDF
Issue
Journal of Hospital Medicine - 6(8)
Publications
Page Number
475-479
Legacy Keywords
cardiac arrest, clinical judgment, rapid response team, track and trigger
Sections
Files
Files
Article PDF
Article PDF

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1
Patient Acuity Rating scale. Abbreviations: ICU, intensive care unit.

Study Setting and Participants

This study was conducted at The University of Chicago Medical Center, an academic, tertiary care facility with approximately 600 inpatient beds. Subjects involved both the clinicians who provided PAR scores and the patients upon whom the PAR scores and outcomes were based. The clinicians included internal medicine interns, residents, and attending physicians, as well as midlevel providers (nurse practitioners or physician assistants). Clinicians were eligible for inclusion if they cared for patients on one of nine adult ward services between January and June 2008. They were included in the study if they consented to participate. Housestaff, with medicine attending supervision, covered patients on seven general medicine services, while midlevel practitioners, also with medicine attending supervision, covered patients on two hepatology and transplant services.

Providers were independently surveyed once per call‐cycle (every 2 to 4 days depending on the service) by study personnel regarding each of their patients, and instructed not to consult with other members of the team regarding their PAR score assignments. All patients for whom a participating clinician provided a PAR score were included in the analysis. Clinician subjects were carefully surveyed at the end of their work day, just prior to, or immediately following, their handover to the cross‐covering physician, so as to minimize the risk that they might alter their plan and transfer a patient to the ICU based on the PAR score.

Data Analysis

PAR scores were entered into a database (Excel, Microsoft Corporation, Redmond, WA) and then linked to patient demographic and outcome data obtained from hospital administrative databases. Weighted kappa statistics were used to evaluate inter‐rater reliability. Ordinal trend testing was used to correlate the PAR with patient outcomes by provider. In addition, receiver operator characteristics (ROC) curves were constructed, and area under the curve (AUC) calculated and compared among providers using paired chi‐squared statistics. Sensitivities and specificities were determined for each theoretical PAR cutoff. Clustered multivariate logistic regression was used to adjust for provider, service, and individual patient. All calculations were performed using a statistical software application (Stata, College Station, TX).

Approval

The study protocol, consent, and data collection mechanisms were approved by the Institutional Review Board of the University of Chicago Medical Center. Waiver of consent provisions were used for patients on the basis of minimal harm and general impracticability, while a written consent process was used for patient care providers. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 regulations.

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

Patient and Admission Characteristics
CharacteristicGeneral Medicine Teaching ServicesMultispecialty Non‐Teaching ServicesP‐Value
  • Abbreviation: N/A, not applicable.

Patients (n)1,373290NA
Admissions (n)1,660359NA
Age, mean (SD) years57 (21)57 (13)0.73
Women, n (%)1,006 (61)173 (48)<0.001
Race, n (%)  <0.001
White203 (12)133 (37) 
Black1,129 (68)125 (35) 
Hispanic26 (2)34 (9) 
Asian11 (1)10 (3) 
Other/unknown291 (18)57 (16) 
Severity of illness, n (%)  <0.001
Minor121 (7)2 (1) 
Moderate461 (28)44 (12) 
Major677 (41)179 (50) 
Extreme329 (20)123 (34) 
N/A77 (4)11 (3) 
Discharged home, n (%)1,347 (81)282 (79)0.25
Expired (not hospice), n (%)25 (2)28 (8)<0.001

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Weighted Kappa Statistics by Provider Pair
Provider PairObservations (n)Agreement (%)Weighted Kappa
Interns vs residents1,00687.10.42
Residents vs attendings1,01282.50.35
Interns vs attendings1,02684.40.32
Midlevels vs attendings20885.00.43

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Patient Acuity Rating (PAR) Sensitivities and Specificities
PARAll Patients (n)Decompensating Patients (n)Sensitivity (%)Specificity (%)
7401216.299.2
61843040.595.4
55614662.284.6
41,1206182.468.3
32,0446993.241.0
23,0057398.612.3
13,41974100.00.0
Area Under the Patient Acuity Rating (PAR) Receiver Operator Characteristics Curve by Provider
ServiceProviderObservations (n)PAR, median (IQR)AUROC (95% CI)
  • Abbreviations: AUROC, area under the receiver operator characteristics curve; CI, confidence interval; IQR, interquartile range.

  • The only significant difference in pair‐wise comparison occurred between residents and attendings (P = 0.01).

General medicineInterns1,5673 (2‐4)0.79 (0.70, 0.88)
General medicineResidents1,6113 (2‐4)0.69 (0.59, 0.78)*
General medicineAttendings1,7913 (2‐4)0.84 (0.78, 0.90)*
MultispecialtyAttendings8233 (2‐4)0.88 (0.79, 0.97)
MultispecialtyMidlevels2423 (2‐4)0.80 (0.64, 0.95)
CombinedAll3,4193 (2‐4)0.82 (0.77, 0.87)
Figure 2
Percent of patients showing clinical deterioration to the point of cardiac arrest or intensive care unit (ICU) transfer by Patient Acuity Rating (PAR) and provider. Ordinal trend P‐value was <0.001 for each provider.

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Acknowledgements

The authors are grateful for the support and dedication of the residents, faculty, and staff in the Department of Medicine at the University of Chicago, without whom this study would not have been possible. They also thank Trevor C. Yuen for statistical support, David Beiser, MD, MS, and Kenneth Rasinski, PhD, for scientific advice, Kate Weaver for expert administrative support and Deborah Walsh, RN, MS, Jefferson Cua, and Amanda Schmitz for assistance with data collection.

Recently released Accreditation Council for Graduate Medical Education (ACGME) recommendations, set to take effect in 2011, place further limits on resident duty hours, which are expected to result in additional handoffs of patients between inpatient providers.1 The increase in these handoffs following the prior set of ACGME recommendations in 2003 has been cited as a potential etiology for the underwhelming effects of the duty hour restrictions on patient outcomes, whereby the benefits of well‐rested physicians are theorized to be offset by increased harm associated with discontinuous care, especially in high‐risk patients.2 In 2007, an Institute of Medicine committee on the topic recommended improving handovers to make the transfer of patient responsibility and information more effective and less error prone.3

Several strategies have been proposed, but an ideal way to quickly transfer complex medical information on numerous patients remains to be identified. A standardized metric of a patient's risk level, if accurate, has the potential to summarize how stable or unstable a patient might be. We hypothesized that clinicians would be able to quantify their judgments regarding the stability of their inpatients and that this measure would correlate with impending clinical deterioration as determined by cardiac arrest or intensive care unit (ICU) transfer within the next 24 hours.

METHODS

Study Design

We developed the Patient Acuity Rating (PAR), a 7‐point Likert scale to quantify clinician judgment regarding the stability of inpatients outside the ICU, and conducted a prospective study of its diagnostic accuracy for predicting impending clinical deterioration in an academic tertiary care hospital. Providers were prospectively surveyed once per call‐cycle, on the day after patient admission, and asked to rate each of their patients on their likelihood of suffering a cardiac arrest or being transferred to the ICU. The scale was anchored at both ends, with a PAR of 1 corresponding to extreme unlikelihood of suffering a cardiac arrest or requiring emergent ICU transfer within the next 24 hours, and a PAR of 7 corresponding with extreme likelihood (Figure 1). A score of 4 suggested neither likely nor unlikely to experience an event. No further anchors were provided.

Figure 1
Patient Acuity Rating scale. Abbreviations: ICU, intensive care unit.

Study Setting and Participants

This study was conducted at The University of Chicago Medical Center, an academic, tertiary care facility with approximately 600 inpatient beds. Subjects involved both the clinicians who provided PAR scores and the patients upon whom the PAR scores and outcomes were based. The clinicians included internal medicine interns, residents, and attending physicians, as well as midlevel providers (nurse practitioners or physician assistants). Clinicians were eligible for inclusion if they cared for patients on one of nine adult ward services between January and June 2008. They were included in the study if they consented to participate. Housestaff, with medicine attending supervision, covered patients on seven general medicine services, while midlevel practitioners, also with medicine attending supervision, covered patients on two hepatology and transplant services.

Providers were independently surveyed once per call‐cycle (every 2 to 4 days depending on the service) by study personnel regarding each of their patients, and instructed not to consult with other members of the team regarding their PAR score assignments. All patients for whom a participating clinician provided a PAR score were included in the analysis. Clinician subjects were carefully surveyed at the end of their work day, just prior to, or immediately following, their handover to the cross‐covering physician, so as to minimize the risk that they might alter their plan and transfer a patient to the ICU based on the PAR score.

Data Analysis

PAR scores were entered into a database (Excel, Microsoft Corporation, Redmond, WA) and then linked to patient demographic and outcome data obtained from hospital administrative databases. Weighted kappa statistics were used to evaluate inter‐rater reliability. Ordinal trend testing was used to correlate the PAR with patient outcomes by provider. In addition, receiver operator characteristics (ROC) curves were constructed, and area under the curve (AUC) calculated and compared among providers using paired chi‐squared statistics. Sensitivities and specificities were determined for each theoretical PAR cutoff. Clustered multivariate logistic regression was used to adjust for provider, service, and individual patient. All calculations were performed using a statistical software application (Stata, College Station, TX).

Approval

The study protocol, consent, and data collection mechanisms were approved by the Institutional Review Board of the University of Chicago Medical Center. Waiver of consent provisions were used for patients on the basis of minimal harm and general impracticability, while a written consent process was used for patient care providers. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 regulations.

RESULTS

During the study period, 140/159 (88.1%) eligible clinicians consented to participate. Of these clinicians, 45 (32.1%) were intern physicians, 40 (28.6%) were resident physicians, 51 (36.4%) were attending physicians, and 4 (2.9%) were midlevel providers. They provided PAR scores on 1663 distinct patients over the course of 2019 separate admissions. Table 1 shows the patient and admission demographics grouped by the type of medical service: general medicine teaching or multispecialty non‐teaching. Severity of illness assignments were determined using All Patient Refined Diagnosis Related Group (APR‐DRG) methodology, which incorporates features such as principle diagnosis at admission, co‐morbidities, complications during admission, age, and gender.4, 5 The multispecialty patients were more likely to be male, have a higher severity of illness, and die during the hospitalization, when compared to general medicine patients.

Patient and Admission Characteristics
CharacteristicGeneral Medicine Teaching ServicesMultispecialty Non‐Teaching ServicesP‐Value
  • Abbreviation: N/A, not applicable.

Patients (n)1,373290NA
Admissions (n)1,660359NA
Age, mean (SD) years57 (21)57 (13)0.73
Women, n (%)1,006 (61)173 (48)<0.001
Race, n (%)  <0.001
White203 (12)133 (37) 
Black1,129 (68)125 (35) 
Hispanic26 (2)34 (9) 
Asian11 (1)10 (3) 
Other/unknown291 (18)57 (16) 
Severity of illness, n (%)  <0.001
Minor121 (7)2 (1) 
Moderate461 (28)44 (12) 
Major677 (41)179 (50) 
Extreme329 (20)123 (34) 
N/A77 (4)11 (3) 
Discharged home, n (%)1,347 (81)282 (79)0.25
Expired (not hospice), n (%)25 (2)28 (8)<0.001

A total of 6034 individual PAR scores from 3419 patient‐days were obtained, which represented a response rate of 74.3%. The average PAR was 2.9 1.4. Table 2 shows the inter‐rater reliability between providers. Weighted kappa statistics ranged from 0.32 (for interns and attendings) to 0.43 (for midlevels and attendings), representing moderate inter‐rater reliability. No comparison was made between midlevel providers and interns or residents, as these participants never cared for the same patients on the same day.

Weighted Kappa Statistics by Provider Pair
Provider PairObservations (n)Agreement (%)Weighted Kappa
Interns vs residents1,00687.10.42
Residents vs attendings1,01282.50.35
Interns vs attendings1,02684.40.32
Midlevels vs attendings20885.00.43

Seventy‐four of the 3419 patient‐days (2.2%) ended in cardiac arrest or unplanned ICU transfer. The distribution of clinical deterioration by average PAR, along with sensitivity and specificity values, are shown in Table 3. Using a cutoff value of 5 yielded a sensitivity of 62.2% and a specificity of 84.6%. Lowering the threshold to 4 increased the sensitivity to 82.4% but decreased the specificity to 68.3%. This corresponded with a combined AUC of 0.82 [95% CI 0.77, 0.87] (Table 4). Provider‐specific AUC values ranged from a low of 0.69 [95% CI 0.59, 0.78] for residents to a high of 0.84 [95% CI 0.78, 0.90] for attendings on general medicine (P = 0.01). The remaining values were not statistically different from one another. Figure 2 shows the provider‐specific percentage of patients deteriorating by PAR. The risk of clinical deterioration decreased in logarithmic fashion as the PAR decreased for all provider types (P < 0.001). These results were confirmed using multivariate logistic regression adjusting for provider, service, and individual patient (data not shown). In addition, we found no significant differences in AUC values between attendings in terms of years in practice or specialty, however, the study was not powered to detect such differences.

Patient Acuity Rating (PAR) Sensitivities and Specificities
PARAll Patients (n)Decompensating Patients (n)Sensitivity (%)Specificity (%)
7401216.299.2
61843040.595.4
55614662.284.6
41,1206182.468.3
32,0446993.241.0
23,0057398.612.3
13,41974100.00.0
Area Under the Patient Acuity Rating (PAR) Receiver Operator Characteristics Curve by Provider
ServiceProviderObservations (n)PAR, median (IQR)AUROC (95% CI)
  • Abbreviations: AUROC, area under the receiver operator characteristics curve; CI, confidence interval; IQR, interquartile range.

  • The only significant difference in pair‐wise comparison occurred between residents and attendings (P = 0.01).

General medicineInterns1,5673 (2‐4)0.79 (0.70, 0.88)
General medicineResidents1,6113 (2‐4)0.69 (0.59, 0.78)*
General medicineAttendings1,7913 (2‐4)0.84 (0.78, 0.90)*
MultispecialtyAttendings8233 (2‐4)0.88 (0.79, 0.97)
MultispecialtyMidlevels2423 (2‐4)0.80 (0.64, 0.95)
CombinedAll3,4193 (2‐4)0.82 (0.77, 0.87)
Figure 2
Percent of patients showing clinical deterioration to the point of cardiac arrest or intensive care unit (ICU) transfer by Patient Acuity Rating (PAR) and provider. Ordinal trend P‐value was <0.001 for each provider.

DISCUSSION

Physicians frequently depend on subjective judgments in their decision making.6 However, these judgments are difficult to communicate succinctly and hard to compare among clinicians. We have developed a simple tool for quantifying provider judgment, which yields moderate inter‐rater reliability, and good accuracy in predicting which floor patients may suffer cardiac arrest or emergent ICU transfer in the next 24 hours at an academic medical center.

Physicians routinely use written sign‐outs to convey important information to covering physicians during the handoff process, with the result being loss of information and decreased communication.7, 8 A recent study found that sign‐outs are frequently lacking comprehensive data, with the least commonly conveyed information being the patient's current clinical condition.9 The PAR has the potential to improve clinician handoffs by succinctly summarizing a patient's risk level. This need is made even more pressing by the ACGME's new resident duty hour restrictions and impending further increase in handoffs, a known correlate with inpatient morbidity and mortality.10 The PAR could be added to the sign‐out and updated nightly to readily summarize the judgments of the primary inpatient providers for the covering physician who has little, if any, personal knowledge of the patient at hand.

While ours is the first to examine the correlation between physician judgment and clinical deterioration on the floors, several studies have evaluated the accuracy of clinical judgment in predicting mortality of critically ill hospitalized patients. In the study to understand prognoses and preferences for outcomes and risks of treatments (SUPPORT), for example, physicians accurately predicted 180‐day mortality of ICU patients with an AUC of 0.78.11 Similar studies in the neonatal intensive care setting revealed greater than 80% predictive ability to identify those patients who would be intubated or fail to survive.12 These intuitive judgments have faired well when compared to previously validated ICU‐based physiologic scoring systems, such as the Acute Physiology, Age, Chronic Health Evaluation (APACHE) system. A meta‐analysis of studies which compared physician intuition to various physiologic scoring systems found intuition to be significantly better at predicting ICU mortality with an AUC of 0.85 compared to 0.63, P = 0.002.13

Physiology‐based scoring systems, relying on routine vital signs, have been developed for non‐ICU inpatients. Smith and colleagues14 recently conducted a systematic review and identified 33 distinct scoring systems, which they independently validated on a single data set for the ability of the admission score to predict overall hospital mortality. The resulting AUC values ranged from 0.66 to 0.78.14 In a prospective study, Kho and colleagues used an electronic medical record (EMR) to generate real‐time risk scores for use in surveillance rounds by their Rapid Response Team (RRT).15 Their scoring system relied on systolic blood pressure, heart rate, respiratory rate, temperature, oxygen saturation, age, and body mass index. The resulting score yielded an AUC of 0.72, with a sensitivity of 88% and specificity of 48% using a cutoff of 3, or 34% and 86% using a cutoff of 5, for predicting code call, cardiopulmonary arrest, or transfer to an ICU. Similar to the latter study, using the PAR for RRT surveillance would allow an institution to set its threshold according to available resources. For example, the team could first evaluate all the patients with a PAR score of 7, followed by those who received a score of 6 and so on. Using the data from the current study, evaluating all the patients with scores of 5, 6, or 7 would require assessing 16% of the patients in order to identify 62% of the events. Adding patients with a score of 4 would require assessing one‐third of the floor patients, but would identify 82% of subsequent deteriorations.

Although the objective nature of physiology‐based scoring systems makes them very appealing, they have two significant limitations. The first is that these scoring systems either require manual vital sign data entry and score calculation, which can be labor intensive and impractical, or technological solutions such as an EMR, which are costly and therefore cannot be applied broadly to all hospitalized patients. In fact, in a recent survey of U.S. hospitals, only 1.5% were found to have a comprehensive EMR on all units.16 Additionally, they are limited by the quality of the data input. This is particularly true for the case of respiratory rate and mental status, which are frequently unreliably measured and documented in current practice.17, 18 The PAR score has the benefit of being readily generated in minimal time by a broad range of providers, as we have demonstrated.

Furthermore, it is well known that vital signs do not capture the full clinical picture, which is why most RRT activation criteria include a vague catch‐all trigger for provider worry or concern.19, 20 Interestingly, this trigger is frequently one of the top cited reasons for activating the RRT,21, 22 and is missed by any automated track‐and‐trigger scoring system which relies only on quantitative clinical assessments such as vital signs. The PAR allows this concern to be quantified, either for addition to a physiology‐based track‐and‐trigger system, or for use on its own, as we have done here.

It is interesting to note that, in this study, attending physician judgment was most predictive and resident judgment the least. One explanation may be that clinical judgment optimally requires both experience and at‐the‐bedside data. While attendings have the most experience, the amount of time interns spend at the bedside collecting data may offset their relative inexperience.23, 24 In contrast, residents generally spend less time at the bedside than interns and have only marginally more experience,25 suggesting that either strong clinical experience or a good amount of time at the bedside are required for the best assessments of risk. This is supported by the close agreement between the attendings and midlevels, who likely spend a comparable amount of time at the bedside as interns.

There are several imitations to this study. First, there may be respondent bias in those who chose to participate and the days in which they provided scores. We would expect this bias to work against the null hypothesis if providers with better clinical judgment were more inclined to participate, and were less likely to provide scores when they were very busy, and thus may have had less time to assess patients. However, the enrollment and response rates were quite good (88% and 74%, respectively) which likely mitigates against this bias. Another limitation is that the study was conducted at a single institution, and only on medical patients, which may limit its generalizability to other institutions and patient populations. Also, intern performance during this January through June period may not reflect their performance earlier in their training year. In addition, we did not have physiologic data available for the patients, and thus were not able to compare the PAR directly to a physiology‐based scoring system. Finally, it is theoretically possible that a provider could decide on the PAR and then transfer the patient to the ICU based on their score. However, we carefully surveyed physicians and nurse practitioners at the time of sign‐out, when they had finished their clinical work for the day, to minimize this risk. We also instructed providers not to share their PAR score with the covering physicians to avoid introduction of bias on the part of the cross‐covering physician.

This was a pilot study designed to measure the correlation between PAR scores and patient outcomes. The PAR has the potential to be added to any handoff system as a way to convey individual severity of illness for patients. In addition, it has the potential for use in risk stratifying patients for interventions, such as increased vital sign monitoring or heightened surveillance by cross‐covering physicians or Rapid Response Teams. One could imagine instructing interns to have a low threshold of concern for patients with high PAR scores, and even formalizing procedures for rounding on such patients a second time during the day or overnight, when on call. Future studies will be required to test its use in clinical practice, which would ideally include a randomized‐controlled trial.

We conclude that clinical judgment regarding floor patient stability is quantifiable in a readily obtained, low‐technology score that has moderate inter‐rater reliability and a good ability to distinguish patients who will suffer a cardiac arrest or require ICU transfer within the next 24 hours. Due to its simple and easy to administer nature, the PAR has the potential to be a useful tool for efficiently conveying complex assessments from one member of the healthcare team to another, thereby improving handoffs and identifying patients at risk of clinical deterioration who might benefit from earlier intervention.

Acknowledgements

The authors are grateful for the support and dedication of the residents, faculty, and staff in the Department of Medicine at the University of Chicago, without whom this study would not have been possible. They also thank Trevor C. Yuen for statistical support, David Beiser, MD, MS, and Kenneth Rasinski, PhD, for scientific advice, Kate Weaver for expert administrative support and Deborah Walsh, RN, MS, Jefferson Cua, and Amanda Schmitz for assistance with data collection.

References
  1. Accreditation Council for Graduate Medical Education (ACGME).Task Force on Quality Care and Professionalism.Proposed standards: common program requirements.2007. Available at: http://acgme‐2010standards.org/pdf/Proposed_Standards.pdf. Accessed August 16,year="2010"2010.
  2. Arora VM,Georgitis E,Siddique J, et al.Association of workload of on‐call medical interns with on‐call sleep duration, shift duration, and participation in educational activities.JAMA.2008;300(10):11461153.
  3. Institute of Medicine (IOM).Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety.Report brief on resident duty hours: enhancing sleep, supervision, and safety.Washington, DC:National Academy Press;2008. Available at: http://www.iom.edu/∼/media/Files/Report%20Files/2008/Resident‐Duty‐Hours/residency%20hours%20revised% 20for%20web.pdf. Accessed August 16,year="2010"2010.
  4. Kuo PC,Douglas AR,Oleski D,Jacobs DO,Schroeder RA.Determining benchmarks for evaluation and management coding in an academic division of general surgery.J Am Coll Surg.2004;199(1):124130.
  5. Shen YJ.Applying the 3M all patient refined diagnosis related groups grouper to measure inpatient severity in the VA.Med Care.2003;41(6):103110.
  6. Norman G,Young M,Brooks L.Non‐analytical models of clinical reasoning: the role of experience.Med Educ.2007;41(12):11401145.
  7. Arora V,Johnson J,Lovinger D,Humphrey HJ,Meltzer DO.Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis.Qual Saf Health Care.2005;14(6):401407.
  8. Gandhi TK.Fumbled handoffs: one dropped ball after another.Ann Intern Med.2005;142(5):352358.
  9. Horwitz LI,Moin T,Krumholz HM,Wang L,Bradley EH.What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff.Qual Saf Health Care.2009;18(4):248255.
  10. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  11. Knaus WA,Harrell FE,Lynn J, et al.The SUPPORT prognostic model—objective estimates of survival for seriously ill hospitalized adults.Ann Intern Med.1995;122(3):191203.
  12. Meadow W,Frain L,Ren YY,Lee G,Soneji S,Lantos J.Serial assessment of mortality in the neonatal intensive care unit by algorithm and intuition: certainty, uncertainty, and informed consent.Pediatrics.2002;109(5):878886.
  13. Sinuff T,Adhikari NKJ,Cook DJ, et al.Mortality predictions in the intensive care unit: comparing physicians with scoring systems.Crit Care Med.2006;34(3):878885.
  14. Smith GB,Prytherch DR,Schmidt PL,Featherstone PI.Review and performance evaluation of aggregate weighted ‘track and trigger’ systems.Resuscitation.2008;77(2):170179.
  15. Kho A,Rotz D,Alrahi K, et al.Utility of commonly captured data from an EHR to identify hospitalized patients at risk for clinical deterioration.AMIA Annu Symp Proc.2007:404408.
  16. Jha AK,DesRoches CM,Campbell EG, et al.Use of electronic health records in U.S. hospitals.N Engl J Med.2009;360(16):16281638.
  17. Lovett PB,Buchwald JM,Sturmann K,Bijur P.The vexatious vital: neither clinical measurements by nurses nor an electronic monitor provides accurate measurements of respiratory rate in triage.Ann Emerg Med.2005;45(1):6876.
  18. Laurila JV,Pitkala KH,Strandberg TE,Tilvis RS.Detection and documentation of dementia and delirium in acute geriatric wards.Gen Hosp Psychiatry.2004;26(1):3135.
  19. Hillman K,Parr M,Flabouris A,Bishop G,Stewart A.Redefining in‐hospital resuscitation: the concept of the medical emergency team.Resuscitation.2001;48(2):105110.
  20. Hodgetts TJ,Kenward G,Vlachonikolis IG,Payne S,Castle N.The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team.Resuscitation.2002;54(2):125131.
  21. Brilli RJ,Gibson R,Luria JW, et al.Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit.Pediatr Crit Care Med.2007;8(3):236246.
  22. Tibballs J,Kinney S,Duke T,Oakley E,Hennessy M.Reduction of paediatric in‐patient cardiac arrest and death with a medical emergency team: preliminary results.Arch Dis Child.2005;90(11):11481152.
  23. Guarisco S,Oddone E,Simel D.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9(5):272277.
  24. Nerenz D,Rosman H,Newcomb C, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150(11):22942297.
  25. Dresselhaus TR,Luck J,Wright BC,Spragg RG,Lee ML,Bozzette SA.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13(8):534540.
References
  1. Accreditation Council for Graduate Medical Education (ACGME).Task Force on Quality Care and Professionalism.Proposed standards: common program requirements.2007. Available at: http://acgme‐2010standards.org/pdf/Proposed_Standards.pdf. Accessed August 16,year="2010"2010.
  2. Arora VM,Georgitis E,Siddique J, et al.Association of workload of on‐call medical interns with on‐call sleep duration, shift duration, and participation in educational activities.JAMA.2008;300(10):11461153.
  3. Institute of Medicine (IOM).Committee on Optimizing Graduate Medical Trainee (Resident) Hours and Work Schedules to Improve Patient Safety.Report brief on resident duty hours: enhancing sleep, supervision, and safety.Washington, DC:National Academy Press;2008. Available at: http://www.iom.edu/∼/media/Files/Report%20Files/2008/Resident‐Duty‐Hours/residency%20hours%20revised% 20for%20web.pdf. Accessed August 16,year="2010"2010.
  4. Kuo PC,Douglas AR,Oleski D,Jacobs DO,Schroeder RA.Determining benchmarks for evaluation and management coding in an academic division of general surgery.J Am Coll Surg.2004;199(1):124130.
  5. Shen YJ.Applying the 3M all patient refined diagnosis related groups grouper to measure inpatient severity in the VA.Med Care.2003;41(6):103110.
  6. Norman G,Young M,Brooks L.Non‐analytical models of clinical reasoning: the role of experience.Med Educ.2007;41(12):11401145.
  7. Arora V,Johnson J,Lovinger D,Humphrey HJ,Meltzer DO.Communication failures in patient sign‐out and suggestions for improvement: a critical incident analysis.Qual Saf Health Care.2005;14(6):401407.
  8. Gandhi TK.Fumbled handoffs: one dropped ball after another.Ann Intern Med.2005;142(5):352358.
  9. Horwitz LI,Moin T,Krumholz HM,Wang L,Bradley EH.What are covering doctors told about their patients? Analysis of sign‐out among internal medicine house staff.Qual Saf Health Care.2009;18(4):248255.
  10. Petersen LA,Brennan TA,O'Neil AC,Cook EF,Lee TH.Does housestaff discontinuity of care increase the risk for preventable adverse events?Ann Intern Med.1994;121(11):866872.
  11. Knaus WA,Harrell FE,Lynn J, et al.The SUPPORT prognostic model—objective estimates of survival for seriously ill hospitalized adults.Ann Intern Med.1995;122(3):191203.
  12. Meadow W,Frain L,Ren YY,Lee G,Soneji S,Lantos J.Serial assessment of mortality in the neonatal intensive care unit by algorithm and intuition: certainty, uncertainty, and informed consent.Pediatrics.2002;109(5):878886.
  13. Sinuff T,Adhikari NKJ,Cook DJ, et al.Mortality predictions in the intensive care unit: comparing physicians with scoring systems.Crit Care Med.2006;34(3):878885.
  14. Smith GB,Prytherch DR,Schmidt PL,Featherstone PI.Review and performance evaluation of aggregate weighted ‘track and trigger’ systems.Resuscitation.2008;77(2):170179.
  15. Kho A,Rotz D,Alrahi K, et al.Utility of commonly captured data from an EHR to identify hospitalized patients at risk for clinical deterioration.AMIA Annu Symp Proc.2007:404408.
  16. Jha AK,DesRoches CM,Campbell EG, et al.Use of electronic health records in U.S. hospitals.N Engl J Med.2009;360(16):16281638.
  17. Lovett PB,Buchwald JM,Sturmann K,Bijur P.The vexatious vital: neither clinical measurements by nurses nor an electronic monitor provides accurate measurements of respiratory rate in triage.Ann Emerg Med.2005;45(1):6876.
  18. Laurila JV,Pitkala KH,Strandberg TE,Tilvis RS.Detection and documentation of dementia and delirium in acute geriatric wards.Gen Hosp Psychiatry.2004;26(1):3135.
  19. Hillman K,Parr M,Flabouris A,Bishop G,Stewart A.Redefining in‐hospital resuscitation: the concept of the medical emergency team.Resuscitation.2001;48(2):105110.
  20. Hodgetts TJ,Kenward G,Vlachonikolis IG,Payne S,Castle N.The identification of risk factors for cardiac arrest and formulation of activation criteria to alert a medical emergency team.Resuscitation.2002;54(2):125131.
  21. Brilli RJ,Gibson R,Luria JW, et al.Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit.Pediatr Crit Care Med.2007;8(3):236246.
  22. Tibballs J,Kinney S,Duke T,Oakley E,Hennessy M.Reduction of paediatric in‐patient cardiac arrest and death with a medical emergency team: preliminary results.Arch Dis Child.2005;90(11):11481152.
  23. Guarisco S,Oddone E,Simel D.Time analysis of a general medicine service: results from a random work sampling study.J Gen Intern Med.1994;9(5):272277.
  24. Nerenz D,Rosman H,Newcomb C, et al.The on‐call experience of interns in internal medicine. Medical Education Task Force of Henry Ford Hospital.Arch Intern Med.1990;150(11):22942297.
  25. Dresselhaus TR,Luck J,Wright BC,Spragg RG,Lee ML,Bozzette SA.Analyzing the time and value of housestaff inpatient work.J Gen Intern Med.1998;13(8):534540.
Issue
Journal of Hospital Medicine - 6(8)
Issue
Journal of Hospital Medicine - 6(8)
Page Number
475-479
Page Number
475-479
Publications
Publications
Article Type
Display Headline
Patient acuity rating: Quantifying clinical judgment regarding inpatient stability
Display Headline
Patient acuity rating: Quantifying clinical judgment regarding inpatient stability
Legacy Keywords
cardiac arrest, clinical judgment, rapid response team, track and trigger
Legacy Keywords
cardiac arrest, clinical judgment, rapid response team, track and trigger
Sections
Article Source

Copyright © 2011 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Section of Hospital Medicine, University of Chicago, 5841 S. Maryland Avenue, MC 5000, Chicago, IL 60637
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media
Media Files