Vincent X Liu, MD, MS

Adults admitted to general medical-surgical wards who experience in-hospital deterioration have a disproportionate effect on hospital mortality and length of stay.¹ Not long ago, systematic electronic capture of vital signs—arguably the most important predictors of impending deterioration—was restricted to intensive care units (ICUs). Deployment of comprehensive electronic health records (EHRs) and handheld charting tools have made vital signs data more accessible, expanding the possibilities of early detection.

In this issue, Peelen et al² report their scoping review of contemporary EHR-based predictive models for identifying ward patients at risk for deterioration. They identified 22 publications suitable for review. Impressively, some studies report extraordinary statistical performance, with positive predictive values (PPVs) exceeding 50% and with 12- to 24-hour lead times to prepare a clinician response. However, only five algorithms were implemented in an EHR and only three were used clinically. Peelen et al also quantified 48 barriers to and 54 facilitators of the implementation and use of these models. Improved statistical performance (higher PPVs) compared to manually assigned scores were the most important facilitators, while implementation in the context of daily practice (alarm fatigue, integration with existing workflows) were the most important barriers.

These reports invite an obvious question: If the models are this good, why have we not seen more reports of improved patient outcomes? Based on our own recent experience successfully deploying and evaluating the Advance Alert Monitor Program for early detection in a 21-hospital system,³ we suspect that there are several factors at play. Despite the relative computational ease of developing high-performing predictive models, it can be very challenging to create the right dataset (extracting and formatting data, standardizing variable definitions across different EHR builds). Investigators may also underestimate the difficulty of what can be implemented—and sustained—in real-world clinical practice. We encountered substantial difficulty, for example, around alarm fatigue mitigation and the relationship of alerts to end-of-life decisions. Greater attention to implementation is necessary to advance the field.

We suggest that four critical questions be considered when creating in-hospital predictive models. First, what are the statistical characteristics of a model around the likely clinical decision point? Simply having a high C-statistic is insufficient—what matters is the alert’s PPV at a clinically actionable threshold.⁴ Second, workflow burden—how many alerts per day at my hospital—must be measured, including other processes potentially affected by the new system. Third, will the extra work identify a meaningful proportion of the avoidable bad outcomes? Finally, how will model use affect care of patients near the end of life? Alerts for these patients may not make clinical sense and might even interfere with overall care (eg, by triggering an unwanted ICU transfer).

Implementation requires more than data scientists. Consideration must be given to system governance, predictive model maintenance (models can actually decalibrate over time!), and financing (not just the computation side—someone needs to pay for training clinicians and ensuring proper staffing of the clinical response).

Last, rigorous model evaluation must be undertaken. Given the increasing capabilities of comprehensive EHRs, patient-level randomization is becoming more feasible. But even randomized deployments present challenges. Since ward patients are a heterogeneous population, quantifying process-outcome relationships may be difficult. Alternative approaches to quantification of the impact of bundled interventions may need to be considered—not just for initial deployment, but on an ongoing basis. Peelen et al² have effectively summarized the state of published predictive models, which hold the tantalizing possibility of meaningful improvement: saved lives, decreased morbidity. Now, we must work together to address the identified gaps so that, one day, implementation of real-time models is routine, and the promise of in-hospital predictive analytics is fulfilled.

Adults admitted to general medical-surgical wards who experience in-hospital deterioration have a disproportionate effect on hospital mortality and length of stay.¹ Not long ago, systematic electronic capture of vital signs—arguably the most important predictors of impending deterioration—was restricted to intensive care units (ICUs). Deployment of comprehensive electronic health records (EHRs) and handheld charting tools have made vital signs data more accessible, expanding the possibilities of early detection.

In this issue, Peelen et al² report their scoping review of contemporary EHR-based predictive models for identifying ward patients at risk for deterioration. They identified 22 publications suitable for review. Impressively, some studies report extraordinary statistical performance, with positive predictive values (PPVs) exceeding 50% and with 12- to 24-hour lead times to prepare a clinician response. However, only five algorithms were implemented in an EHR and only three were used clinically. Peelen et al also quantified 48 barriers to and 54 facilitators of the implementation and use of these models. Improved statistical performance (higher PPVs) compared to manually assigned scores were the most important facilitators, while implementation in the context of daily practice (alarm fatigue, integration with existing workflows) were the most important barriers.

These reports invite an obvious question: If the models are this good, why have we not seen more reports of improved patient outcomes? Based on our own recent experience successfully deploying and evaluating the Advance Alert Monitor Program for early detection in a 21-hospital system,³ we suspect that there are several factors at play. Despite the relative computational ease of developing high-performing predictive models, it can be very challenging to create the right dataset (extracting and formatting data, standardizing variable definitions across different EHR builds). Investigators may also underestimate the difficulty of what can be implemented—and sustained—in real-world clinical practice. We encountered substantial difficulty, for example, around alarm fatigue mitigation and the relationship of alerts to end-of-life decisions. Greater attention to implementation is necessary to advance the field.

We suggest that four critical questions be considered when creating in-hospital predictive models. First, what are the statistical characteristics of a model around the likely clinical decision point? Simply having a high C-statistic is insufficient—what matters is the alert’s PPV at a clinically actionable threshold.⁴ Second, workflow burden—how many alerts per day at my hospital—must be measured, including other processes potentially affected by the new system. Third, will the extra work identify a meaningful proportion of the avoidable bad outcomes? Finally, how will model use affect care of patients near the end of life? Alerts for these patients may not make clinical sense and might even interfere with overall care (eg, by triggering an unwanted ICU transfer).

Implementation requires more than data scientists. Consideration must be given to system governance, predictive model maintenance (models can actually decalibrate over time!), and financing (not just the computation side—someone needs to pay for training clinicians and ensuring proper staffing of the clinical response).

Last, rigorous model evaluation must be undertaken. Given the increasing capabilities of comprehensive EHRs, patient-level randomization is becoming more feasible. But even randomized deployments present challenges. Since ward patients are a heterogeneous population, quantifying process-outcome relationships may be difficult. Alternative approaches to quantification of the impact of bundled interventions may need to be considered—not just for initial deployment, but on an ongoing basis. Peelen et al² have effectively summarized the state of published predictive models, which hold the tantalizing possibility of meaningful improvement: saved lives, decreased morbidity. Now, we must work together to address the identified gaps so that, one day, implementation of real-time models is routine, and the promise of in-hospital predictive analytics is fulfilled.

Adults admitted to general medical-surgical wards who experience in-hospital deterioration have a disproportionate effect on hospital mortality and length of stay.¹ Not long ago, systematic electronic capture of vital signs—arguably the most important predictors of impending deterioration—was restricted to intensive care units (ICUs). Deployment of comprehensive electronic health records (EHRs) and handheld charting tools have made vital signs data more accessible, expanding the possibilities of early detection.

In this issue, Peelen et al² report their scoping review of contemporary EHR-based predictive models for identifying ward patients at risk for deterioration. They identified 22 publications suitable for review. Impressively, some studies report extraordinary statistical performance, with positive predictive values (PPVs) exceeding 50% and with 12- to 24-hour lead times to prepare a clinician response. However, only five algorithms were implemented in an EHR and only three were used clinically. Peelen et al also quantified 48 barriers to and 54 facilitators of the implementation and use of these models. Improved statistical performance (higher PPVs) compared to manually assigned scores were the most important facilitators, while implementation in the context of daily practice (alarm fatigue, integration with existing workflows) were the most important barriers.

These reports invite an obvious question: If the models are this good, why have we not seen more reports of improved patient outcomes? Based on our own recent experience successfully deploying and evaluating the Advance Alert Monitor Program for early detection in a 21-hospital system,³ we suspect that there are several factors at play. Despite the relative computational ease of developing high-performing predictive models, it can be very challenging to create the right dataset (extracting and formatting data, standardizing variable definitions across different EHR builds). Investigators may also underestimate the difficulty of what can be implemented—and sustained—in real-world clinical practice. We encountered substantial difficulty, for example, around alarm fatigue mitigation and the relationship of alerts to end-of-life decisions. Greater attention to implementation is necessary to advance the field.

We suggest that four critical questions be considered when creating in-hospital predictive models. First, what are the statistical characteristics of a model around the likely clinical decision point? Simply having a high C-statistic is insufficient—what matters is the alert’s PPV at a clinically actionable threshold.⁴ Second, workflow burden—how many alerts per day at my hospital—must be measured, including other processes potentially affected by the new system. Third, will the extra work identify a meaningful proportion of the avoidable bad outcomes? Finally, how will model use affect care of patients near the end of life? Alerts for these patients may not make clinical sense and might even interfere with overall care (eg, by triggering an unwanted ICU transfer).

Implementation requires more than data scientists. Consideration must be given to system governance, predictive model maintenance (models can actually decalibrate over time!), and financing (not just the computation side—someone needs to pay for training clinicians and ensuring proper staffing of the clinical response).

Last, rigorous model evaluation must be undertaken. Given the increasing capabilities of comprehensive EHRs, patient-level randomization is becoming more feasible. But even randomized deployments present challenges. Since ward patients are a heterogeneous population, quantifying process-outcome relationships may be difficult. Alternative approaches to quantification of the impact of bundled interventions may need to be considered—not just for initial deployment, but on an ongoing basis. Peelen et al² have effectively summarized the state of published predictive models, which hold the tantalizing possibility of meaningful improvement: saved lives, decreased morbidity. Now, we must work together to address the identified gaps so that, one day, implementation of real-time models is routine, and the promise of in-hospital predictive analytics is fulfilled.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.