In Reply: We thank Dr. Moskowitz for his kind comments. We agree about the need for assessing vitamin B₁₂ levels during chronic metformin use.

Secondary analysis of patients in the Diabetes Prevention Program Outcomes Study showed a higher incidence of combined low and low-normal vitamin B₁₂ deficiency in users assigned to the metformin group compared with those assigned to the placebo group at the 5-year and 13-year marks after randomization.1 Post hoc analysis of patients in the Hyperinsulinemia: the Outcome of Its Metabolic Effects trial also showed lower levels of vitamin B₁₂ and higher levels of methylmalonic acid associated with significant worsening of a validated neuropathy score in metformin users.²

The mechanism behind the development of vitamin B₁₂ deficiency is not completely understood but could possibly be alterations in intestinal mobility, bacterial overgrowth, or calcium-dependent uptake by ileal cells of the vitamin B₁₂-intrinsic factor complex.³

Our electronic medical record has a built-in tool that suggests checking vitamin B₁₂ whenever a patient requests metformin refills. There are no current guidelines on the need for baseline testing of the vitamin B₁₂ level. The American Diabetes Association recommends periodic measurement of vitamin B₁₂ levels, possibly yearly, in metformin users and more often if there are symptoms indicative of deficiency.⁴

In Reply: We thank Dr. Moskowitz for his kind comments. We agree about the need for assessing vitamin B₁₂ levels during chronic metformin use.

Secondary analysis of patients in the Diabetes Prevention Program Outcomes Study showed a higher incidence of combined low and low-normal vitamin B₁₂ deficiency in users assigned to the metformin group compared with those assigned to the placebo group at the 5-year and 13-year marks after randomization.1 Post hoc analysis of patients in the Hyperinsulinemia: the Outcome of Its Metabolic Effects trial also showed lower levels of vitamin B₁₂ and higher levels of methylmalonic acid associated with significant worsening of a validated neuropathy score in metformin users.²

The mechanism behind the development of vitamin B₁₂ deficiency is not completely understood but could possibly be alterations in intestinal mobility, bacterial overgrowth, or calcium-dependent uptake by ileal cells of the vitamin B₁₂-intrinsic factor complex.³

Our electronic medical record has a built-in tool that suggests checking vitamin B₁₂ whenever a patient requests metformin refills. There are no current guidelines on the need for baseline testing of the vitamin B₁₂ level. The American Diabetes Association recommends periodic measurement of vitamin B₁₂ levels, possibly yearly, in metformin users and more often if there are symptoms indicative of deficiency.⁴

In Reply: We thank Dr. Moskowitz for his kind comments. We agree about the need for assessing vitamin B₁₂ levels during chronic metformin use.

Secondary analysis of patients in the Diabetes Prevention Program Outcomes Study showed a higher incidence of combined low and low-normal vitamin B₁₂ deficiency in users assigned to the metformin group compared with those assigned to the placebo group at the 5-year and 13-year marks after randomization.1 Post hoc analysis of patients in the Hyperinsulinemia: the Outcome of Its Metabolic Effects trial also showed lower levels of vitamin B₁₂ and higher levels of methylmalonic acid associated with significant worsening of a validated neuropathy score in metformin users.²

The mechanism behind the development of vitamin B₁₂ deficiency is not completely understood but could possibly be alterations in intestinal mobility, bacterial overgrowth, or calcium-dependent uptake by ileal cells of the vitamin B₁₂-intrinsic factor complex.³

Our electronic medical record has a built-in tool that suggests checking vitamin B₁₂ whenever a patient requests metformin refills. There are no current guidelines on the need for baseline testing of the vitamin B₁₂ level. The American Diabetes Association recommends periodic measurement of vitamin B₁₂ levels, possibly yearly, in metformin users and more often if there are symptoms indicative of deficiency.⁴

Here are 5 articles from the April issue of Clinician Reviews (individual articles are valid for one year from date of publication—expiration dates below):

1. Back pain persists in one in five patients

To take the posttest, go to: https://bit.ly/2Uiod8N
Expires January 14, 2019

2. COPD linked to higher in-hospital death rates in patients with PAD

To take the posttest, go to: https://bit.ly/2TFCeJC
Expires January 22, 2019

3. Medicaid youth suicides include more females, younger kids, hanging deaths

To take the posttest, go to: https://bit.ly/2Uleyyp
Expires January 17, 2019

4. Potential antidepressant overprescribing found in 24% of elderly cohort

To take the posttest, go to: https://bit.ly/2HWwcSq
Expires January 24, 2019

5. Perceptions of liver transplantation for ALD are evolving

To take the posttest, go to: https://bit.ly/2OCANuA
Expires January 22, 2019

Here are 5 articles from the April issue of Clinician Reviews (individual articles are valid for one year from date of publication—expiration dates below):

1. Back pain persists in one in five patients

To take the posttest, go to: https://bit.ly/2Uiod8N
Expires January 14, 2019

2. COPD linked to higher in-hospital death rates in patients with PAD

To take the posttest, go to: https://bit.ly/2TFCeJC
Expires January 22, 2019

3. Medicaid youth suicides include more females, younger kids, hanging deaths

To take the posttest, go to: https://bit.ly/2Uleyyp
Expires January 17, 2019

4. Potential antidepressant overprescribing found in 24% of elderly cohort

To take the posttest, go to: https://bit.ly/2HWwcSq
Expires January 24, 2019

5. Perceptions of liver transplantation for ALD are evolving

To take the posttest, go to: https://bit.ly/2OCANuA
Expires January 22, 2019

Here are 5 articles from the April issue of Clinician Reviews (individual articles are valid for one year from date of publication—expiration dates below):

1. Back pain persists in one in five patients

To take the posttest, go to: https://bit.ly/2Uiod8N
Expires January 14, 2019

2. COPD linked to higher in-hospital death rates in patients with PAD

To take the posttest, go to: https://bit.ly/2TFCeJC
Expires January 22, 2019

3. Medicaid youth suicides include more females, younger kids, hanging deaths

To take the posttest, go to: https://bit.ly/2Uleyyp
Expires January 17, 2019

4. Potential antidepressant overprescribing found in 24% of elderly cohort

To take the posttest, go to: https://bit.ly/2HWwcSq
Expires January 24, 2019

5. Perceptions of liver transplantation for ALD are evolving

To take the posttest, go to: https://bit.ly/2OCANuA
Expires January 22, 2019

Both CPR rates and survival rates increased after chest compression–only CPR was introduced as an alternative for bystanders witnessing a cardiac arrest, according to a Swedish study of out-of-hospital cardiac arrests and subsequent CPR.

© American Heart Association, Inc.

“These findings support continuous endorsement of chest compression–only CPR as an option in future CPR guidelines because it is associated with higher CPR rates and survival in out-of-hospital cardiac arrests,” wrote Gabriel Riva, MD, of the Karolinska Institutet, Stockholm, and his coauthors. The study was published in Circulation.

To determine changes in the rate and type of CPR performed before emergency medical services (EMS) arrival, the researchers compared all bystander-witnessed out-of-hospital cardiac arrests (OHCAs) reported in Sweden between 2000 and 2017. In all, 30,445 patients were included; the time periods compared were 2000-2005, 2006-2010, and 2011-2017. Patients were categorized as receiving either no CPR (NO-CPR), standard CPR (S-CPR), or chest compression–only CPR (CO-CPR). In 2005, CO-CPR was introduced in national CPR guidelines as an option for bystanders; in 2010, it was recommended for anyone untrained in CPR.

The proportion of patients who received CPR in general increased from 41% in 2000-2005 to 59% in 2006-2010 to 68% in 2011-2017. S-CPR changed from 35% to 45% to 38% over the three periods, while CO-CPR increased from 5% to 14% to 30%. In regard to 30-day survival rates, the S-CPR group saw an increase from 9% to 13% to 16% and the CO-CPR group increased from 8% to 12% to 14%, compared with 4% to 6% to 7% for the NO-CPR group.

The authors noted the limitations of their study, including the results being based on register data and therefore subject to misclassification and missing data. In addition, missing data negated any reporting on the neurological function of survivors; analyzing witnessed OHCAs only also meant the findings could not be validated for nonwitnessed OHCA.

The Swedish Heart and Lung Foundation funded the study. The authors made no disclosures.

SOURCE: Riva G et al. Circulation. 2019 Apr 1. doi: 10.1161/CIRCULATIONAHA.118.038179.

Both CPR rates and survival rates increased after chest compression–only CPR was introduced as an alternative for bystanders witnessing a cardiac arrest, according to a Swedish study of out-of-hospital cardiac arrests and subsequent CPR.

© American Heart Association, Inc.

“These findings support continuous endorsement of chest compression–only CPR as an option in future CPR guidelines because it is associated with higher CPR rates and survival in out-of-hospital cardiac arrests,” wrote Gabriel Riva, MD, of the Karolinska Institutet, Stockholm, and his coauthors. The study was published in Circulation.

To determine changes in the rate and type of CPR performed before emergency medical services (EMS) arrival, the researchers compared all bystander-witnessed out-of-hospital cardiac arrests (OHCAs) reported in Sweden between 2000 and 2017. In all, 30,445 patients were included; the time periods compared were 2000-2005, 2006-2010, and 2011-2017. Patients were categorized as receiving either no CPR (NO-CPR), standard CPR (S-CPR), or chest compression–only CPR (CO-CPR). In 2005, CO-CPR was introduced in national CPR guidelines as an option for bystanders; in 2010, it was recommended for anyone untrained in CPR.

The proportion of patients who received CPR in general increased from 41% in 2000-2005 to 59% in 2006-2010 to 68% in 2011-2017. S-CPR changed from 35% to 45% to 38% over the three periods, while CO-CPR increased from 5% to 14% to 30%. In regard to 30-day survival rates, the S-CPR group saw an increase from 9% to 13% to 16% and the CO-CPR group increased from 8% to 12% to 14%, compared with 4% to 6% to 7% for the NO-CPR group.

The authors noted the limitations of their study, including the results being based on register data and therefore subject to misclassification and missing data. In addition, missing data negated any reporting on the neurological function of survivors; analyzing witnessed OHCAs only also meant the findings could not be validated for nonwitnessed OHCA.

The Swedish Heart and Lung Foundation funded the study. The authors made no disclosures.

SOURCE: Riva G et al. Circulation. 2019 Apr 1. doi: 10.1161/CIRCULATIONAHA.118.038179.

Both CPR rates and survival rates increased after chest compression–only CPR was introduced as an alternative for bystanders witnessing a cardiac arrest, according to a Swedish study of out-of-hospital cardiac arrests and subsequent CPR.

© American Heart Association, Inc.

“These findings support continuous endorsement of chest compression–only CPR as an option in future CPR guidelines because it is associated with higher CPR rates and survival in out-of-hospital cardiac arrests,” wrote Gabriel Riva, MD, of the Karolinska Institutet, Stockholm, and his coauthors. The study was published in Circulation.

To determine changes in the rate and type of CPR performed before emergency medical services (EMS) arrival, the researchers compared all bystander-witnessed out-of-hospital cardiac arrests (OHCAs) reported in Sweden between 2000 and 2017. In all, 30,445 patients were included; the time periods compared were 2000-2005, 2006-2010, and 2011-2017. Patients were categorized as receiving either no CPR (NO-CPR), standard CPR (S-CPR), or chest compression–only CPR (CO-CPR). In 2005, CO-CPR was introduced in national CPR guidelines as an option for bystanders; in 2010, it was recommended for anyone untrained in CPR.

The proportion of patients who received CPR in general increased from 41% in 2000-2005 to 59% in 2006-2010 to 68% in 2011-2017. S-CPR changed from 35% to 45% to 38% over the three periods, while CO-CPR increased from 5% to 14% to 30%. In regard to 30-day survival rates, the S-CPR group saw an increase from 9% to 13% to 16% and the CO-CPR group increased from 8% to 12% to 14%, compared with 4% to 6% to 7% for the NO-CPR group.

The authors noted the limitations of their study, including the results being based on register data and therefore subject to misclassification and missing data. In addition, missing data negated any reporting on the neurological function of survivors; analyzing witnessed OHCAs only also meant the findings could not be validated for nonwitnessed OHCA.

The Swedish Heart and Lung Foundation funded the study. The authors made no disclosures.

SOURCE: Riva G et al. Circulation. 2019 Apr 1. doi: 10.1161/CIRCULATIONAHA.118.038179.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.

Delirium is an acute change in mental status, affecting more than seven million hospitalized patients in the United States annually.¹ Several factors increase the risk of developing delirium, including advanced age,² cognitive dysfunction,³ hearing and vision impairment,^4-6 and severe illness or major surgery.⁷ Delirium may be precipitated during hospitalization by common inpatient interventions, such as the use of physical restraints, polypharmacy, or bladder catheters.^4,8 In-hospital delirium impacts an estimated 10%-15% of the general medical admissions and as many as 81% of patients in the intensive care unit (ICU).^9-11 Despite the relative frequency with which delirium is encountered in the hospital, subsequent emergency department (ED) presentations or hospital readmissions for these patients are poorly characterized.

The development of delirium is associated with several negative outcomes during the hospital stay. Delirium is an independent predictor of prolonged hospital stay,^7,9,12,13 prolonged mechanical ventilation,¹⁴ and mortality during admission.^14,15 Inpatient delirium is associated with functional decline at discharge, leading to a new nursing home placement.^16-19 Preexisting dementia is exacerbated by inpatient delirium, and a new diagnosis of cognitive impairment²⁰ or dementia becomes more common after an episode of delirium.²¹

These data suggest that people diagnosed with delirium may be particularly vulnerable in the posthospitalization period. Hospitals with high rates of unplanned readmissions face penalties from the Centers for Medicare and Medicaid Services.^22,23 However, few investigations have focused on postdischarge healthcare utilization, such as readmission rates and ED visits. Studies that address this topic are limited to postoperative patient populations.²⁴

Using a cohort of hospitalized patients, we examined whether those diagnosed with delirium experienced worse outcomes compared with patients with no such condition. We hypothesized that the patients diagnosed with delirium during hospitalization would experience more readmissions and ED visits within 30 days of discharge compared with those without delirium.

Study Design

This single-center retrospective cohort study took place at the Kaiser Permanente San Rafael Medical Center (KP-SRF), a 116-bed general community medical and surgical hospital located in Northern California, from September 6, 2010 to March 31, 2015. The Kaiser Permanente Northern California institutional review board, in accordance with the provisions of the Declaration of the Helsinki and International Conference on Harmonization Guidelines for Good Clinical Practice (CN-15-2491-H), approved this study.

Participants and Eligibility Criteria

This study included Kaiser Permanente members at least 65 years old who were hospitalized at KP-SRF from September 2010 to March 2015. Patient data were obtained from the electronic medical records. Patients with delirium were identified from a delirium registry; all other patients served as controls.

Starting on September 6, 2010, a hospital-wide program was initiated to screen hospitalized medical and surgical patients using the Confusion Assessment Method (CAM).²⁵ As part of this program, nurses completed a four-hour training on delirium; the program included delirium identification and CAM administration. Patients deemed at risk for delirium by their nurse or displaying symptoms of delirium (fluctuation in attention or awareness, disorientation, restlessness, agitation, and psychomotor slowing) were screened by nurses one to two times within a 24-hour period. Physicians were notified by the nurse if their patient screened positive. Nurses were prohibited from performing CAMs in languages that they were not fluent in, thus resulting in screening of primarily English-speaking patients. Psychiatry was consulted at the discretion of the primary team physician to assist with diagnosis and management of delirium. As psychiatry consultation was left up to the discretion of the primary team physician, not all CAM-positive patients were evaluated. The psychiatrists conducted no routine evaluation on the CAM-negative patients unless requested by the primary team physician. The psychiatrist confirmed the delirium diagnosis with a clinical interview and assessment. The patients confirmed with delirium at any point during their hospitalization were prospectively added to a delirium registry. The patients assessed by the psychiatrist as not delirious were excluded from the registry. Only those patients added to the delirium registry during the study period were classified as delirious for this study. All other patients were included as controls. The presence of the nursing screening program using the CAM enriched the cohort, but a positive CAM was unnecessary nor was it sufficient for inclusion in the delirium group (Table 1).

Patient Characteristics

Patient demographics and clinical data were obtained from the electronic medical records. We used several scores to characterize illness severity, including the Charlson comorbidity index,²⁷ Laboratory-Based Acute Physiology, version 2 (LAPS2) score²⁸—an externally validated score for acute severity of illness—and disease categories as defined by the Healthcare Cost and Utilization Project (HCUP).²⁹

Outcomes

The primary outcome was the rate of readmission to the hospital within 30 days of discharge from the hospitalization in which delirium was first diagnosed. Readmissions and ED visits to any Kaiser Permanente hospital and to hospitals outside of the Kaiser Permanente network with Kaiser Permanente insurance were captured. To avoid incorrectly coding patients transferred from the index hospital to another hospital as readmissions, we excluded readmissions that occurred on the day of discharge or the following calendar day. This action was expected to lower the absolute number of readmissions but restrict the analysis to true readmissions. The models of postdischarge outcomes are based on the subset of patients discharged alive. The secondary outcome measures included discharge from the index hospitalization to a skilled nursing facility or hospice rather than to home and emergency room visits within 30 days of discharge. We also quantified rates of mortality during hospitalization and at 30 days postdischarge.

Statistical Analysis

Comparisons between patients with delirium and those without were performed using Pearson’s X² test for categorical variables and student t-test for continuous variables. The estimated odds of our outcome measures for delirious and nondelirious subjects were calculated from multivariable logistic regression models, which controlled for predictors of delirium and additional information obtained during the hospitalization. For inpatient outcomes (in-hospital mortality and discharge to skilled nursing facility or hospice), we adjusted only for admission characteristics: age, race/ethnicity, admission to ICU, Charlson comorbidity index, HCUP category, and admission category. To limit the number of variables in our model, we consolidated the initial 30 HCUP categories (Appendix Table 1) by illness type into 13 categories (Appendix Table 2). For postdischarge outcomes, we adjusted for all the variables, including disposition (Table 2). The average estimated odds were calculated based on the observed marginal distribution of the control variables. The P value indicates how likely the odds on each outcome for delirious subjects differed significantly from those for other subjects. All statistical analyses were performed using SAS 9.3 (SAS Institute Inc., Cary, North Carolina).

Demographics and Clinical Characteristics

A total of 718 patients with delirium and 7,927 patients without delirium were included in this study. The related demographic information is outlined in Table 2. On average, the patients with delirium were older (83 ± 8 years versus 77 ± 8 years, P < .0001) but no difference in gender distribution was observed between groups. A similar racial breakdown was noted between groups, with white patients accounting for 87% of both patients with delirium and those without. The majority of admissions were unplanned medical admissions. The delirium cohort included more emergent surgical admissions compared with patients who did not develop delirium. Patients who developed delirium exhibited higher levels of illness severity on admission, as measured by the Charlson and LAPS2 scores, and were more often admitted to the ICU. Significant differences were also observed between admission illness categories between patients with delirium and those without.

Primary Outcome

Delirium during admission was significantly associated with hospital readmission within 30 days of discharge (adjusted odds ratio [aOR] = 2.60, 95% CI: 1.96–3.44; P < .0001; Table 3).

Secondary Outcomes

Delirium during admission was significantly (P < .0001; Table 3) associated with an ED visit within 30 days of discharge (OR: 2.18; 95% CI: 1.77–2.69) and discharge to a skilled nursing facility or hospice rather than home (OR: 2.52; 95% CI: 2.09–3.01). Delirium was not associated (P > .1) with death during hospitalization nor death 30 days following discharge.

As the delirious patients were much more likely to be discharged to a skilled nursing facility than nondelirious patients, we tested whether discharge disposition influenced readmission rates and ED visits between delirious and nondelirious patients in an unadjusted univariate analysis. The association between delirium and readmission and ED utilization was present regardless of disposition. Among patients discharged to skilled nursing, readmission rates were 4.76% and 13.38% (P < .001), and ED visit rates were 12.29% and 23.24% (P < .001) for nondelirious and delirious patients, respectively. Among patients discharged home, readmission rates were 4.96% and 14.37% (P < .001), and ED visit rates were 11.93% and 29.04% (P < .001) for nondelirious and delirious patients, respectively.

In this study of patients in a community hospital in Northern California, we observed a significant association between inpatient delirium and risk of hospital readmission within 30 days of discharge. We also demonstrated increased skilled nursing facility placement and ED utilization after discharge among hospitalized patients with delirium compared with those without. Patients with delirium in this study were diagnosed by a psychiatrist—a gold standard³⁰—and the study was conducted in a health system database with near comprehensive ascertainment of readmissions. These results suggest that patients with delirium are particularly vulnerable in the posthospitalization period and are a key group to focusing on reducing readmission rates and postdischarge healthcare utilization.

Identifying the risk factors for hospital readmission is important for the benefit of both the patient and the hospital. In an analysis of Medicare claims data from 2003 to 2004, 19.6% of beneficiaries were readmitted within 30 days of discharge.³¹ There is a national effort to reduce unplanned hospital readmissions for both patient safety as hospitals with high readmission rates face penalties from the Centers for Medicare and Medicaid Services.^22,23 Why delirium is associated with readmission remains unclear. Delirium may precipitate aspiration events, reduce oral intake which complicates medication administration and nutrition, or reduced mobility, leading to pulmonary emboli and skin breakdown, any of which could lead to readmission.³² Delirium may also accelerate the progression of cognitive decline and overall loss of functional independence.²⁰ Delirious patients can be difficult to care for at home, and persistent delirium may lead to returns to the ED and readmission. Strategies to reduce readmissions associated with delirium may need to focus on both prevention of hospital-acquired delirium and targeted caregiver and patient support after discharge.

Hospital readmission and ED visits are not mutually exclusive experiences. In the United States, the majority of patients admitted to the hospital are admitted through the ED.³³ Thus, most of the readmissions in this cohort were also likely counted as 30-day ED visits. However, as ED utilization occurs regardless of whether a patient is discharged or admitted from the ED, we reported all ED visits in this analysis, similar to other studies.³⁴ More delirium patients returned to the ED 30 days postdischarge than were ultimately readmitted to the hospital, and delirious patients were more likely to visit the ED or be readmitted than nondelirious patients. These observations point toward the first 30 days after discharge as a crucial period for these patients.

Our study features several strengths. To our knowledge, this study is one of the largest investigations of inpatients with delirium. One distinguishing feature was that all cases of delirium in this study were diagnosed by a psychiatrist, which is considered a gold standard. Many studies rely solely on brief nursing-administered surveys for delirium diagnosis. Using Kaiser Permanente data allowed for more complete follow-up of patients, including vital status. Kaiser Permanente is both a medical system and an insurer, resulting in acquisition of detailed health information from all hospitalizations where Kaiser Permanente insurance was used for each patient. Therefore, patients were only lost to follow-up following discharge in the event of a membership lapse; these patients were excluded from analysis. The obtained data are also more generalizable than those of other studies examining readmission rates in delirious patients as the hospital where these data were collected is a 116-bed general community medical and surgical hospital. Thus, the patients enrolled in this study covered multiple hospital services with a variety of admission diagnoses. This condition contrasts with much of the existing literature on inpatient delirium; these studies mostly center on specific medical conditions or surgeries and are often conducted at academic medical centers. At the same time, Kaiser Permanente is a unique health maintenance organization focused on preventive care, and readmission rates are possibly lower than elsewhere given the universal access to primary care for Kaiser Permanente members. Our results may not generalize to patients hospitalized in other health systems.

The diagnosis of delirium is a clinical diagnosis without biomarkers or radiographic markers and is also underdiagnosed and poorly coded.³² For these reasons, delirium can be challenging to study in large administrative databases or data derived from electronic medical records. We addressed this limitation by classifying the delirium patients only when they had been diagnosed by a staff psychiatrist. However, not all patients who screened positive with the CAM were evaluated by the staff psychiatrist during the study period. Thus, several CAM-positive patients who were not evaluated by psychiatry were included in the control population. This situation may cause bias toward identification of more severe cases of delirium. Although the physicians were encouraged to consult the psychiatry department for any patients who screened positive for delirium with the CAM, the psychiatrist may not have been involved if patients were managed without consultation. These patients may have exhibited less severe delirium or hypoactive delirium. In addition, the CAM fails to detect all delirious patients; interrater variability may occur with CAM administration, and non-English speaking patients are more likely to be excluded.³⁵ These situations are another possible way for our control population to include some delirious patients and those patients with less severe or hypoactive subtypes. While this might bias toward the null hypothesis, it is also possible our results only indicate an association between more clinically apparent delirium and readmission. A major limitation of this study is that we were unable to quantify the number of cohort patients screened with the CAM or the results of screening, thus limiting our ability to quantify the impact of potential biases introduced by the screening program.

This study may have underestimated readmission rates. We defined readmissions as all hospitalizations at any Kaiser Permanente facility, or to an alternate facility where Kaiser Permanente insurance was used, within 30 days of discharge. We excluded the day of discharge or the following calendar day to avoid mischaracterizing transfers from the index hospital to another Kaiser Permanente facility as readmissions. This step was conducted to avoid biasing our comparison, as delirious patients are less frequently discharged home than nondelirious patients. Therefore, while the relative odds of readmission between delirious and nondelirious patients reported in this study should be generalizable to other community hospitals, the absolute readmission rates reported here may not be comparable to those reported in other studies.

Delirium may represent a marker of more severe illness or medical complications accrued during the hospitalization, which could lead to the associations observed in this study due to confounding.³² Patients with delirium are more likely to be admitted emergently, admitted to the ICU, and feature higher acuity conditions than patients without delirium. We attempted to mitigate this possibility by using a multivariable model to control for variables related to illness severity, including the Charlson comorbidity index, HCUP diagnostic categories, and ICU admission. Despite including HCUP diagnostic categories in our model, we were unable to capture the contribution of certain diseases with finer granularity, such as preexistent dementia, which may also affect clinical outcomes.³⁶ Similarly, although we incorporated markers of illness severity into our model, we were unable to adjust for baseline functional status or frailty, which were not reliably recorded in the electronic medical record but are potential confounders when investigating clinical outcomes including hospital readmission.

We also lacked information regarding the duration of delirium in our cohort. Therefore, we were unable to test whether longer episodes of delirium were more predictive of readmission than shorter episodes.

In-hospital delirium is associated with several negative patient outcomes. Our study demonstrates that delirium predicts 30-day readmission and emergency department utilization after hospital discharge. Bearing in mind that a third of hospital-acquired delirium cases may be preventable,³² hospitals should prioritize interventions to reduce postdischarge healthcare utilization and complications in this particularly vulnerable group.

Acknowledgments

The authors would like to acknowledge Dr. Andrew L. Avins for his guidance with the initial development of this project and Julie Fourie for contributing data to the overall study.

Disclosures

Dr. Liu receives funding from NIH K23GM112018 and NIGMS R35128672. Dr. Josephson receives compensation as the JAMA Neurology Editor in Chief and Continuum Audio Associate Editor. The remaining authors have no conflicts of interest.

Funding

This study was funded by Kaiser Permanente Graduate Medical Education, who approved the design, conduct, and reporting of this study.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Almost all specialties in internal medicine have a sound scientific research base through which clinical practice is informed.¹ For the field of Hospital Medicine (HM), this evidence has largely comprised research generated from fields outside of the specialty. The need to develop, invest, and grow investigators in hospital-based medicine remains unmet as HM and its footprint in hospital systems continue to grow.^2,3

Despite this fact, little is known about the current state of research in HM. A 2014 survey of the members of the Society of Hospital Medicine (SHM) found that research output across the field of HM, as measured on the basis of peer-reviewed publications, was growing.⁴ Since then, however, the numbers of individuals engaged in research activities, their background and training, publication output, or funding sources have not been quantified. Similarly, little is known about which institutions support the development of junior investigators (ie, HM research fellowships), how these programs are funded, and whether or not matriculants enter the field as investigators. These gaps must be measured, evaluated, and ideally addressed through strategic policy and funding initiatives to advance the state of science within HM.

Members of the SHM Research Committee developed, designed, and deployed a survey to improve the understanding of the state of research in HM. In this study, we aimed to establish the baseline of research in HM to enable the measurement of progress through periodic waves of data collection. Specifically, we sought to quantify and describe the characteristics of existing research programs, the sources and types of funding, the number and background of faculty, and the availability of resources for training researchers in HM.

Study Setting and Participants

Given that no defined list, database, or external resource that identifies research programs and contacts in HM exists, we began by creating a strategy to identify and sample adult HM programs and their leaders engaged in research activity. We iteratively developed a two-step approach to maximize inclusivity. First, we partnered with SHM to identify programs and leaders actively engaging in research activities. SHM is the largest professional organization within HM and maintains an extensive membership database that includes the titles, e-mail addresses, and affiliations of hospitalists in the United States, including academic and nonacademic sites. This list was manually scanned, and the leaders of academic and research programs in adult HM were identified by examining their titles (eg, Division Chief, Research Lead, etc.) and academic affiliations. During this step, members of the committee noticed that certain key individuals were either missing, no longer occupying their role/title, or had been replaced by others. Therefore, we performed a second step and asked the members of the SHM Research Committee to identify academic and research leaders by using current personal contacts, publication history, and social networks. We asked members to identify individuals and programs that had received grant funding, were actively presenting research at SHM (or other major national venues), and/or were producing peer-reviewed publications related to HM. These programs were purposefully chosen (ie, over HM programs known for clinical activities) to create an enriched sample of those engaged in research in HM. The research committee performed the “second pass” to ensure that established investigators who may not be accurately captured within the SHM database were included to maximize yield for the survey. Finally, these two sources were merged to ensure the absence of duplicate contacts and the identification of a primary respondent for each affiliate. As a result, a convenience sample of 100 programs and corresponding individuals was compiled for the purposes of this survey.

Survey Development

A workgroup within the SHM Research Committee was tasked to create a survey that would achieve four distinct goals: (1) identify institutions currently engaging in hospital-based research; (2) define the characteristics, including sources of research funding, training opportunities, criteria for promotion, and grant support, of research programs within institutions; (3) understand the prevalence of research fellowship programs, including size, training curricula, and funding sources; and (4) evaluate the productivity and funding sources of HM investigators at each site.

Survey questions that target each of these domains were drafted by the workgroup. Questions were pretested with colleagues outside the workgroup focused on this project (ie, from the main research committee). The instrument was refined and edited to improve the readability and clarity of questions on the basis of the feedback obtained through the iterative process. The revised instrument was then programmed into an online survey administration tool (SurveyMonkey^®) to facilitate electronic dissemination. Finally, the members of the workgroup tested the online survey to ensure functionality. No identifiable information was collected from respondents, and no monetary incentive was offered for the completion of the survey. An invitation to participate in the survey was sent via e-mail to each of the program contacts identified.

Statistical Analysis

Descriptive statistics, including proportions, means, and percentages, were used to tabulate results. All analyses were conducted using Stata 13 MP/SE (StataCorp, College Station, Texas).

Ethical and Regulatory Considerations

The study was reviewed and deemed exempt from regulation by the University of Michigan Institutional Review Board (HUM000138628).

General Characteristics of Research Programs and Faculty

Out of 100 program contacts, 28 (representing 1,586 faculty members) responded and were included in the survey (program response rate = 28%). When comparing programs that did respond with those that did not, a greater proportion of programs in university settings were noted among respondents (79% vs 21%). Respondents represented programs from all regions of the United States, with most representing university-based (79%), university-affiliated (14%) or Veterans Health Administration (VHA; 11%) programs. Most respondents were in leadership roles, including division chiefs (32%), research directors/leads (21%), section chiefs (18%), and related titles, such as program director. Respondents indicated that the total number of faculty members in their programs (including nonclinicians and advance practice providers) varied from eight to 152 (mean [SD] = 57 [36]) members, with physicians representing the majority of faculty members (Table 1).

Among the 1,586 faculty members within the 28 programs, respondents identified 192 faculty members (12%) as currently receiving extra- or intramural support for research activities. Of these faculty, over half (58%) received <25% of effort from intra or extramural sources, and 28 (15%) and 52 (27%) faculty members received 25%-50% or >50% of support for their effort, respectively. The number of investigators who received funding across programs ranged from 0 to 28 faculty members. Compared with the 192 funded investigators, respondents indicated that a larger number of faculty in their programs (n = 656 or 41%) were involved in local quality improvement (QI) efforts. Of the 656 faculty members involved in QI efforts, 241 individuals (37%) were internally funded and received protected time/effort for their work.

Key Attributes of Research Programs

In the evaluation of the amount of total grant funding, respondents from 17 programs indicated that they received $500,000 in annual extra and intramural funding, and those from three programs stated that they received $500,000 to $999,999 in funding. Five respondents indicated that their programs currently received $1 million to $5 million in grant funding, and three reported >$5 million in research support. The sources of research funding included several divisions within the National Institute of Health (NIH, 12 programs), Agency for Healthcare Research and Quality (AHRQ, four programs), foundations (four programs), and internal grants (six programs). Additionally, six programs indicated “other” sources of funding that included the VHA, Patient-Centered Outcomes Research Institute (PCORI), Centers for Medicare and Medicaid Services, Centers for Disease Control (CDC), and industry sources.

A range of grants, including career development awards (11 programs); small grants, such as R21 and R03s (eight programs); R-level grants, including VA merit awards (five programs); program series grants, such as P and U grants (five programs), and foundation grants (eight programs), were reported as types of awards. Respondents from 16 programs indicated that they provided internal pilot grants. Amounts for such grants ranged from <$50,000 (14 programs) to $50,000-$100,000 (two programs).

Research Fellowship Programs/Training Programs

Only five of the 28 surveyed programs indicated that they currently had a research training or fellowship program for developing hospitalist investigators. The age of these programs varied from <1 year to 10 years. Three of the five programs stated that they had two fellows per year, and two stated they had spots for one trainee annually. All respondents indicated that fellows received training on study design, research methods, quantitative (eg, large database and secondary analyses) and qualitative data analysis. In addition, two programs included training in systematic review and meta-analyses, and three included focused courses on healthcare policy. Four of the five programs included training in QI tools, such as LEAN and Six Sigma. Funding for four of the five fellowship programs came from internal sources (eg, department and CTSA). However, two programs added they received some support from extramural funding and philanthropy. Following training, respondents from programs indicated that the majority of their graduates (60%) went on to hybrid research/QI roles (50/50 research/clinical effort), whereas 40% obtained dedicated research investigator (80/20) positions (Table 2).

The 23 institutions without research training programs cited that the most important barrier for establishing such programs was lack of funding (12 programs) and the lack of a pipeline of hospitalists seeking such training (six programs). However, 15 programs indicated that opportunities for hospitalists to gain research training in the form of courses were available internally (eg, courses in the department or medical school) or externally (eg, School of Public Health). Seven programs indicated that they were planning to start a HM research fellowship within the next five years.

Research Faculty

Among the 28 respondents, 15 stated that they have faculty members who conduct research as their main professional activity (ie, >50% effort). The number of faculty members in each program in such roles varied from one to 10. Respondents indicated that faculty members in this category were most often midcareer assistant or associate professors with few full professors. All programs indicated that scholarship in the form of peer-reviewed publications was required for the promotion of faculty. Faculty members who performed research as their main activity had all received formal fellowship training and consequently had dual degrees (MD with MPH or MD, with MSc being the two most common combinations). With respect to clinical activities, most respondents indicated that research faculty spent 10% to 49% of their effort on clinical work. However, five respondents indicated that research faculty had <10% effort on clinical duties (Table 3).

Eleven respondents (39%) identified the main focus of faculty as health service research, where four (14%) identified their main focus as clinical trials. Regardless of funding status, all respondents stated that their faculty were interested in studying quality and process improvement efforts (eg, transitions or readmissions, n = 19), patient safety initiatives (eg, hospital-acquired complications, n = 17), and disease-specific areas (eg, thrombosis, n = 15).

In terms of research output, 12 respondents stated that their research/QI faculty collectively published 11-50 peer-reviewed papers during the academic year, and 10 programs indicated that their faculty published 0-10 papers per year. Only three programs reported that their faculty collectively published 50-99 peer-reviewed papers per year. With respect to abstract presentations at national conferences, 13 programs indicated that they presented 0-10 abstracts, and 12 indicated that they presented 11-50.

In this first survey quantifying research activities in HM, respondents from 28 programs shared important insights into research activities at their institutions. Although our sample size was small, substantial variation in the size, composition, and structure of research programs in HM among respondents was observed. For example, few respondents indicated the availability of training programs for research in HM at their institutions. Similarly, among faculty who focused mainly on research, variation in funding streams and effort protection was observed. A preponderance of midcareer faculty with a range of funding sources, including NIH, AHRQ, VHA, CMS, and CDC was reported. Collectively, these data not only provide a unique glimpse into the state of research in HM but also help establish a baseline of the status of the field at large.

Some findings of our study are intuitive given our sampling strategy and the types of programs that responded. For example, the fact that most respondents for research programs represented university-based or affiliated institutions is expected given the tripartite academic mission. However, even within our sample of highly motivated programs, some findings are surprising and merit further exploration. For example, the observation that some respondents identified HM investigators within their program with <25% in intra- or extramural funding was unexpected. On the other extreme, we were surprised to find that three programs reported >$5 million in research funding. Understanding whether specific factors, such as the availability of experienced mentors within and outside departments or assistance from support staff (eg, statisticians and project managers), are associated with success and funding within these programs are important questions to answer. By focusing on these issues, we will be well poised as a field to understand what works, what does not work, and why.

Likewise, the finding that few programs within our sample offer formal training in the form of fellowships to research investigators represents an improvement opportunity. A pipeline for growing investigators is critical for the specialty that is HM. Notably, this call is not new; rather, previous investigators have highlighted the importance of developing academically oriented hospitalists for the future of the field.⁵ The implementation of faculty scholarship development programs has improved the scholarly output, mentoring activities, and succession planning of academics within HM.^6,7 Conversely, lack of adequate mentorship and support for academic activities remains a challenge and as a factor associated with the failure to produce academic work.⁸ Without a cadre of investigators asking critical questions related to care delivery, the legitimacy of our field may be threatened.

While extrapolating to the field is difficult given the small number of our respondents, highlighting the progress that has been made is important. For example, while misalignment between funding and clinical and research mission persists, our survey found that several programs have been successful in securing extramural funding for their investigators. Additionally, internal funding for QI work appears to be increasing, with hospitalists receiving dedicated effort for much of this work. Innovation in how best to support and develop these types of efforts have also emerged. For example, the University of Michigan Specialist Hospitalist Allied Research Program offers dedicated effort and funding for hospitalists tackling projects germane to HM (eg, ordering of blood cultures for febrile inpatients) that overlap with subspecialists (eg, infectious diseases).⁹ Thus, hospitalists are linked with other specialties in the development of research agendas and academic products. Similarly, the launch of the HOMERUN network, a coalition of investigators who bridge health systems to study problems central to HM, has helped usher in a new era of research opportunities in the specialty.¹⁰ Fundamentally, the culture of HM has begun to place an emphasis on academic and scholarly productivity in addition to clinical prowess.^11-13 Increased support and funding for training programs geared toward innovation and research in HM is needed to continue this mission. The Society for General Internal Medicine, American College of Physicians, and SHM have important roles to play as the largest professional organizations for generalists in this respect. Support for research, QI, and investigators in HM remains an urgent and largely unmet need.

Our study has limitations. First, our response rate was low at 28% but is consistent with the response rates of other surveys of physician groups.¹⁴ Caution in making inferences to the field at large is necessary given the potential for selection and nonresponse bias. However, we expect that respondents are likely biased toward programs actively conducting research and engaged in QI, thus better reflecting the state of these activities in HM. Second, given that we did not ask for any identifying information, we have no way of establishing the accuracy of the data provided by respondents. However, we have no reason to believe that responses would be altered in a systematic fashion. Future studies that link our findings to publicly available data (eg, databases of active grants and funding) might be useful. Third, while our survey instrument was created and internally validated by hospitalist researchers, its lack of external validation could limit findings. Finally, our results vary on the basis of how respondents answered questions related to effort and time allocation given that these measures differ across programs.

In summary, the findings from this study highlight substantial variations in the number, training, and funding of research faculty across HM programs. Understanding the factors behind the success of some programs and the failures of others appears important in informing and growing the research in the field. Future studies that aim to expand survey participation, raise the awareness of the state of research in HM, and identify barriers and facilitators to academic success in HM are needed.

Disclosures

Dr. Chopra discloses grant funding from the Agency for Healthcare Research and Quality (AHRQ), VA Health Services and Research Department, and Centers for Disease Control. Dr. Jones discloses grant funding from AHRQ. All other authors disclose no conflicts of interest.

Acute respiratory illnesses (ARIs), including acute exacerbations of asthma, croup, pneumonia, and bronchiolitis, are among the most common illnesses in childhood.¹ Although most ARIs can be managed in the outpatient setting, hospitalization is common with respiratory illnesses accounting for >425,000 hospitalizations annually.¹ Pneumonia, asthma, and bronchiolitis each rank among the top five reasons for pediatric hospitalization in the United States.¹ Successful efforts to prevent or mitigate the severity of ARIs could have a major impact on child health.

Exposure to secondhand smoke (SHS) is a preventable risk factor for ARI in children, particularly when there is regular exposure in the home.² Chronic exposure to SHS impacts systemic inflammation by suppressing serum interferon-gamma,³ which can lead to increased susceptibility to viral and bacterial infections,⁴ and increasing Th2 (atopic) cytokine expression, which is associated with asthma.⁵ SHS exposure in children has also been linked to diminished lung function.⁶ As a result, SHS exposure is associated with increased ARI susceptibility and severity in children.^7-10

Much research has focused on the clinical impact of SHS exposure on respiratory health in children, but little is known about the impact on patient-reported outcomes, such as health-related quality of life (HRQOL). Patient-reported outcomes help provide a comprehensive evaluation of the effectiveness of healthcare delivery systems. These outcomes are increasingly used by health service researchers to better understand patient and caregiver perspectives.¹¹ Given the known associations between SHS exposure and ARI morbidity, we postulated that regular SHS exposure would also impact HRQOL in children. In this study, we assessed the relationship between SHS exposure and HRQOL within a large, multicenter, prospective cohort of children presenting to the emergency department (ED) and/or hospital with ARI.

Study Population

This study was nested within the Pediatric Respiratory Illness Measurement System (PRIMES) study, a prospective cohort study of children with ARI in the ED and inpatient settings at five tertiary care children’s hospitals within the Pediatric Research in Inpatient Settings Network in Colorado, Pennsylvania, Tennessee, Texas, and Washington. Eligible children were two weeks to 16 years of age hospitalized after presenting to the ED with a primary diagnosis of asthma, croup, bronchiolitis, or pneumonia between July 1, 2014 and June 30, 2016. Because of an anticipated low frequency of croup hospitalizations, we also included children presenting to the ED and then discharged to home with this diagnosis. Children were assigned to a PRIMES diagnosis group based on their final discharge diagnosis. If there was a discrepancy between admission and discharge diagnoses, the discharge diagnosis was used. If a child had more than one discharge diagnosis for a PRIMES condition (eg, acute asthma and pneumonia), we chose the PRIMES condition with the lowest total enrollments overall. If the final discharge diagnosis was not a PRIMES condition, the case was excluded from further analysis. Patients with immunodeficiency, cystic fibrosis, a history of prematurity <32 weeks, chronic neuromuscular disease, cardiovascular disease, pulmonary diseases (other than asthma), and moderate to severe developmental delay were also excluded. Children admitted to intensive care were eligible only if they were transferred to an acute care ward <72 hours following admission. A survey was administered at the time of enrollment that collected information on SHS exposure, HRQOL, healthcare utilization, and demographics. All study procedures were reviewed and approved by the institutional review boards at each of the participating hospitals.

To ascertain SHS exposure, we asked caregivers, “How many persons living in the child’s home smoke?” Responses were dichotomized into non-SHS exposed (0 smokers) and SHS exposed (≥1 smokers). Children with missing data on home SHS exposure were excluded.

Health-Related Quality of Life Outcomes

We estimated HRQOL using the Pediatric Quality of Life (PedsQL^TM) 4.0 Generic Core and Infant Scales. The PedsQL instruments are validated, population HRQOL measures that evaluate the physical, mental, emotional, and social functioning of children two to 18 years old based on self- or caregiver-proxy report.^12-15 These instruments have also shown responsiveness as well as construct and predictive validity in hospitalized children.¹¹ For this study, we focused on the PedsQL physical functioning subscale, which assesses for problems with physical activities (eg, sports activity or exercise, low energy, and hurts or aches) on a five-point Likert scale (never to almost always a problem). Scores range from 0 to 100 with higher scores indicating a better HRQOL. The reported minimal clinically important difference (MCID), defined as the smallest difference in which individuals would perceive a benefit or would necessitate a change in management, for this scale is 4.5 points.^16,17

Children >8 years old were invited to complete the self-report version of the PedsQL. For children <8 years old, and for older children who were unable to complete them, surveys were completed by a parent or legal guardian. Respondents were asked to assess perceptions of their (or their child’s) HRQOL during periods of baseline health (the child’s usual state of health in the month preceding the current illness) and during the acute illness (the child’s state of health at the time of admission) as SHS exposure may influence perceptions of general health and/or contribute to worse outcomes during periods of acute illness.

Covariates collected at the time of enrollment included sociodemographics (child age, gender, race/ethnicity, and caregiver education), and healthcare utilization (caregiver-reported patient visits to a healthcare provider in the preceding six months). Insurance status and level of medical complexity (using the Pediatric Medical Complexity Algorithm)¹⁸ were obtained using the Pediatric Hospital Information System database, an administrative database containing clinical and resource utilization data from >45 children’s hospitals in the United States including all of the PRIMES study hospitals.¹³

Analysis

Descriptive statistics included frequency (%) and mean (standard deviation). Bivariate comparisons according to SHS exposure status were analyzed using chi-squared tests for categorical variables and analysis of variance for continuous variables. Multivariable linear mixed regression models were used to examine associations between home SHS exposure and HRQOL for baseline health and during admission, overall and stratified by diagnosis. Covariates in each model included age, sex, race/ethnicity, caregiver education, and healthcare visits in the preceding six months. We also included a hospital random effect to account for clustering of patients within hospitals and used robust standard errors for inference.

In a secondary analysis to explore potential dose-response effects of SHS exposure, we examined associations between an ordinal exposure variable (0 smokers, 1 smoker, ≥2 smokers) and HRQOL for baseline health and during admission for the acute illness. Because of sample size limitations, diagnosis-specific analyses examining dose-response effects were not conducted.

Study Population

Of the 2,334 children enrolled in the PRIMES study, 25 (1%) respondents did not report on home SHS exposure and were excluded, yielding a final study population of 2,309 children, of whom 728 (32%) had reported home SHS exposure. The study population included 664 children with asthma (mean age seven years [3.5]; 38% with home SHS exposure), 740 with bronchiolitis (mean age 0.7 years [0.5]; 32% with home SHS exposure), 342 with croup (mean age 1.7 [1.1]; 25% with home SHS exposure), and 563 with pneumonia (mean age 4.4 [3.8]; 27% with home SHS exposure; Table 1). Compared with non-SHS-exposed children, those with home SHS exposure tend to be slightly older (3.9 vs 3.4 years, P = .01), more likely to be non-Hispanic Black (29% vs 19%, P < .001), to have a chronic condition (52% vs 41%, P < .001), to come from a household where caregiver(s) did not graduate from college (45% vs 29%, P < .001), and to have public insurance (73% vs 49%, P < .001).

Home SHS Exposure and Health-related Quality of Life

The overall mean HRQOL score for baseline health was 83 (15), with a range across diagnoses of 82 to 87. Compared with non-SHS-exposed children, children with home SHS exposure had a lower mean HRQOL score for baseline health (adjusted mean difference –3.04 [95% CI -4.34, –1.74]). In analyses stratified by diagnosis, baseline health scores were lower for SHS-exposed children for all four conditions, but differences were statistically significant only for bronchiolitis (adjusted mean difference –2.94 [–5.0, –0.89]) and pneumonia (adjusted mean value –4.13 [–6.82, –1.44]; Table 2); none of these differences met the MCID threshold.

The overall mean HRQOL score at the time of admission was 56 (23), with a range across diagnoses of 49 to 61, with lower scores noted among SHS-exposed children compared with non-SHS-exposed children (adjusted mean difference –2.16 [–4.22, –0.10]). Similar to scores representing baseline health, admission scores were lower across all four conditions for SHS-exposed children. Only children with croup, however, had significantly lower admission scores that also met the MCID threshold (adjusted mean difference –5.71 [–10.67, –0.75]; Table 2).

To assess for potential dose-response effects of SHS exposure on HRQOL, we stratified SHS-exposed children into those with one smoker in the home (n = 513) and those with ≥2 smokers in the home (n = 215). Compared with non-SHS-exposed children, both HRQOL scores (baseline health and admission) were lower for SHS-exposed children. Consistent with a dose-response association, scores were lowest for children with ≥2 smokers in the home, both at baseline health (adjusted mean difference –3.92 [–6.03, –1.81]) and on admission (adjusted mean difference –3.67 [–6.98, –0.36]; Table 3).

Within a multicenter cohort of 2,309 children hospitalized with ARI, we noted significantly lower HRQOL scores among children exposed to SHS in the home as compared with nonexposed children. Differences were greatest for children living with ≥2 smokers in the home. In analyses stratified by diagnosis, differences in baseline health HRQOL scores were greatest for children with bronchiolitis and pneumonia. Differences in acute illness scores were greatest for children with croup.¹⁶

Our study provides evidence for acute and chronic impacts of SHS on HRQOL in children hospitalized with ARI. Although several studies have linked SHS exposure to reduced HRQOL in adults,^19,20 few similar studies have been conducted in children. Nonetheless, a wealth of studies have documented the negative impact of SHS exposure on clinical outcomes among children with ARI.^8,10,21-23 Our findings that home SHS exposure was associated with reduced HRQOL among our cohort of children with ARI are therefore consistent with related findings in adults and children. The observation that the effects of SHS exposure on HRQOL were greatest among children living with ≥2 smokers provides further evidence of a potential causal link between regular SHS exposure and HRQOL.

Although the magnitude and significance of associations between SHS exposure and HRQOL varied for each of the four diagnoses for baseline health and the acute illness, it is important to note that the point estimates for the adjusted mean differences were uniformly lower for the SHS-exposed children in each subgroup. Even so, only acute illness scores for croup exceeded the MCID threshold.¹⁶ Croup is the only included condition of the upper airway and is characterized by laryngotracheal inflammation leading to the typical cough and, in moderate to severe cases, stridor. Given that chronic SHS exposure induces a proinflammatory state,³ it is possible that SHS-exposed children with croup had more severe illness compared with nonexposed children with croup resulting in lower HRQOL scores on admission. Further, perceived differences in illness severity and HRQOL may be more readily apparent in children with croup (eg, stridor at rest vs intermittent or no stridor) as compared with children with lower respiratory tract diseases.

Of the four included diagnoses, the link between SHS exposure and asthma outcomes has been most studied. Prior work has demonstrated more frequent and severe acute exacerbations, as well as worse long-term lung function among SHS-exposed children as compared with nonexposed children.^22-24 It was, therefore, surprising that our study failed to demonstrate associations between SHS exposure and HRQOL among children with asthma. Reasons for this finding are unclear. One hypothesis is that caregivers of SHS-exposed children with asthma may be more aware of the impacts of SHS exposure on respiratory health (through prior education) and, thus, more likely to modify their smoking behaviors, or for their children to be on daily asthma controller therapy. Alternatively, caregivers of children with asthma may be more likely to underreport home SHS exposure. Thirty-eight percent of children with asthma, however, were classified as SHS-exposed. This percentage was greater than the other three conditions studied (25%-32%), suggesting that differential bias in underreporting was minimal. Given that children with asthma were older, on average, than children with the other three conditions, it may also be that these children spent more time in smoke-free environments (eg, school).

Nearly one-third of children in our study were exposed to SHS in the home. This is similar to the prevalence of exposure in other studies conducted among hospitalized children^8,10,21,25 but higher than the national prevalence of home SHS exposure among children in the United States.²⁶ Thus, hospitalized children represent a particularly vulnerable population and an important target for interventions aiming to reduce exposure to SHS. Although longitudinal interventions are likely necessary to affect long-term success, hospitalization for ARI may serve as a powerful teachable moment to begin cessation efforts. Hospitalization also offers time beyond a typical primary care outpatient encounter to focus on cessation counseling and may be the only opportunity to engage in counseling activities for some families with limited time or access. Further, prior studies have demonstrated both the feasibility and the effectiveness of smoking cessation interventions in hospitalized children.^27-30 Unfortunately, however, SHS exposure is often not documented at the time of hospitalization, and many opportunities to intervene are missed.^25,31 Thus, there is a need for improved strategies to reliably identify and intervene on SHS-exposed children in the hospital setting.

These findings should be considered in the context of several limitations. The observational nature of our study raises the potential for confounding, specifically with regard to socioeconomic status, as this is associated with both SHS exposure and lower HRQOL. Our modeling approach attempted to control for several factors associated with socioeconomic status, including caregiver education and insurance coverage, but there is potential for residual confounding. No single question is sufficient to fully assess SHS exposure as the intensity of home SHS exposure likely varies widely, and some children may be exposed to SHS outside of the home environment.³² The home, however, is often the most likely source of regular SHS exposure,^33,34 especially among young children (our cohort’s mean age was 3.6 years). Misclassification of SHS exposure is also possible due to underreporting of smoking.^35,36 As a result, some children regularly exposed to SHS may have been misclassified as nonexposed, and the observed associations between SHS exposure and HRQOL may be underestimated. Confirming our study’s findings using objective assessments of SHS exposure, such as cotinine, are warranted. Given the young age of our cohort, the PedsQL surveys were completed by the parent or legal guardian only in >90% of the enrolled subjects, and caregiver perceptions may not accurately reflect the child’s perceptions. Prior work, however, has demonstrated the validity of parent-proxy reporting of the PedsQL, including correlation with child self-report.³⁷ In our study, correlation between child and caregiver reporting (when available) was also very good (r = 0.72, 95% CI 0.64, 0.77). It is also possible that the timing of the HRQOL assessments (on admission) may have biased perceptions of baseline HRQOL, although we anticipate any bias would likely be nondifferential between SHS-exposed and nonexposed children and across diagnoses.

Nearly one-third of children in our study were exposed to SHS exposure in the home, and SHS exposure was associated with lower HRQOL for baseline health and during acute illness, providing further evidence of the dangers of SHS. Much work is needed in order to eliminate the impact of SHS on child health and families of children hospitalized for respiratory illness should be considered a priority population for smoking cessation efforts.

Acknowledgment

The authors wish to acknowledge the efforts of PRIS-PRIMES study team. The authors also wish to thank the children and families who consented to be a part of the PRIMES study.

Disclosures

The authors have no conflicts of interest relevant to this article to disclose.

Funding

This study was supported by NIH-NHLBI 1R01HL121067 to RMS.

Acute respiratory illnesses (ARIs), including acute exacerbations of asthma, croup, pneumonia, and bronchiolitis, are among the most common illnesses in childhood.¹ Although most ARIs can be managed in the outpatient setting, hospitalization is common with respiratory illnesses accounting for >425,000 hospitalizations annually.¹ Pneumonia, asthma, and bronchiolitis each rank among the top five reasons for pediatric hospitalization in the United States.¹ Successful efforts to prevent or mitigate the severity of ARIs could have a major impact on child health.

Exposure to secondhand smoke (SHS) is a preventable risk factor for ARI in children, particularly when there is regular exposure in the home.² Chronic exposure to SHS impacts systemic inflammation by suppressing serum interferon-gamma,³ which can lead to increased susceptibility to viral and bacterial infections,⁴ and increasing Th2 (atopic) cytokine expression, which is associated with asthma.⁵ SHS exposure in children has also been linked to diminished lung function.⁶ As a result, SHS exposure is associated with increased ARI susceptibility and severity in children.^7-10

Much research has focused on the clinical impact of SHS exposure on respiratory health in children, but little is known about the impact on patient-reported outcomes, such as health-related quality of life (HRQOL). Patient-reported outcomes help provide a comprehensive evaluation of the effectiveness of healthcare delivery systems. These outcomes are increasingly used by health service researchers to better understand patient and caregiver perspectives.¹¹ Given the known associations between SHS exposure and ARI morbidity, we postulated that regular SHS exposure would also impact HRQOL in children. In this study, we assessed the relationship between SHS exposure and HRQOL within a large, multicenter, prospective cohort of children presenting to the emergency department (ED) and/or hospital with ARI.

Study Population

This study was nested within the Pediatric Respiratory Illness Measurement System (PRIMES) study, a prospective cohort study of children with ARI in the ED and inpatient settings at five tertiary care children’s hospitals within the Pediatric Research in Inpatient Settings Network in Colorado, Pennsylvania, Tennessee, Texas, and Washington. Eligible children were two weeks to 16 years of age hospitalized after presenting to the ED with a primary diagnosis of asthma, croup, bronchiolitis, or pneumonia between July 1, 2014 and June 30, 2016. Because of an anticipated low frequency of croup hospitalizations, we also included children presenting to the ED and then discharged to home with this diagnosis. Children were assigned to a PRIMES diagnosis group based on their final discharge diagnosis. If there was a discrepancy between admission and discharge diagnoses, the discharge diagnosis was used. If a child had more than one discharge diagnosis for a PRIMES condition (eg, acute asthma and pneumonia), we chose the PRIMES condition with the lowest total enrollments overall. If the final discharge diagnosis was not a PRIMES condition, the case was excluded from further analysis. Patients with immunodeficiency, cystic fibrosis, a history of prematurity <32 weeks, chronic neuromuscular disease, cardiovascular disease, pulmonary diseases (other than asthma), and moderate to severe developmental delay were also excluded. Children admitted to intensive care were eligible only if they were transferred to an acute care ward <72 hours following admission. A survey was administered at the time of enrollment that collected information on SHS exposure, HRQOL, healthcare utilization, and demographics. All study procedures were reviewed and approved by the institutional review boards at each of the participating hospitals.

To ascertain SHS exposure, we asked caregivers, “How many persons living in the child’s home smoke?” Responses were dichotomized into non-SHS exposed (0 smokers) and SHS exposed (≥1 smokers). Children with missing data on home SHS exposure were excluded.

Health-Related Quality of Life Outcomes

We estimated HRQOL using the Pediatric Quality of Life (PedsQL^TM) 4.0 Generic Core and Infant Scales. The PedsQL instruments are validated, population HRQOL measures that evaluate the physical, mental, emotional, and social functioning of children two to 18 years old based on self- or caregiver-proxy report.^12-15 These instruments have also shown responsiveness as well as construct and predictive validity in hospitalized children.¹¹ For this study, we focused on the PedsQL physical functioning subscale, which assesses for problems with physical activities (eg, sports activity or exercise, low energy, and hurts or aches) on a five-point Likert scale (never to almost always a problem). Scores range from 0 to 100 with higher scores indicating a better HRQOL. The reported minimal clinically important difference (MCID), defined as the smallest difference in which individuals would perceive a benefit or would necessitate a change in management, for this scale is 4.5 points.^16,17

Children >8 years old were invited to complete the self-report version of the PedsQL. For children <8 years old, and for older children who were unable to complete them, surveys were completed by a parent or legal guardian. Respondents were asked to assess perceptions of their (or their child’s) HRQOL during periods of baseline health (the child’s usual state of health in the month preceding the current illness) and during the acute illness (the child’s state of health at the time of admission) as SHS exposure may influence perceptions of general health and/or contribute to worse outcomes during periods of acute illness.

Covariates collected at the time of enrollment included sociodemographics (child age, gender, race/ethnicity, and caregiver education), and healthcare utilization (caregiver-reported patient visits to a healthcare provider in the preceding six months). Insurance status and level of medical complexity (using the Pediatric Medical Complexity Algorithm)¹⁸ were obtained using the Pediatric Hospital Information System database, an administrative database containing clinical and resource utilization data from >45 children’s hospitals in the United States including all of the PRIMES study hospitals.¹³

Analysis

Descriptive statistics included frequency (%) and mean (standard deviation). Bivariate comparisons according to SHS exposure status were analyzed using chi-squared tests for categorical variables and analysis of variance for continuous variables. Multivariable linear mixed regression models were used to examine associations between home SHS exposure and HRQOL for baseline health and during admission, overall and stratified by diagnosis. Covariates in each model included age, sex, race/ethnicity, caregiver education, and healthcare visits in the preceding six months. We also included a hospital random effect to account for clustering of patients within hospitals and used robust standard errors for inference.

In a secondary analysis to explore potential dose-response effects of SHS exposure, we examined associations between an ordinal exposure variable (0 smokers, 1 smoker, ≥2 smokers) and HRQOL for baseline health and during admission for the acute illness. Because of sample size limitations, diagnosis-specific analyses examining dose-response effects were not conducted.

Study Population

Of the 2,334 children enrolled in the PRIMES study, 25 (1%) respondents did not report on home SHS exposure and were excluded, yielding a final study population of 2,309 children, of whom 728 (32%) had reported home SHS exposure. The study population included 664 children with asthma (mean age seven years [3.5]; 38% with home SHS exposure), 740 with bronchiolitis (mean age 0.7 years [0.5]; 32% with home SHS exposure), 342 with croup (mean age 1.7 [1.1]; 25% with home SHS exposure), and 563 with pneumonia (mean age 4.4 [3.8]; 27% with home SHS exposure; Table 1). Compared with non-SHS-exposed children, those with home SHS exposure tend to be slightly older (3.9 vs 3.4 years, P = .01), more likely to be non-Hispanic Black (29% vs 19%, P < .001), to have a chronic condition (52% vs 41%, P < .001), to come from a household where caregiver(s) did not graduate from college (45% vs 29%, P < .001), and to have public insurance (73% vs 49%, P < .001).

Home SHS Exposure and Health-related Quality of Life

The overall mean HRQOL score for baseline health was 83 (15), with a range across diagnoses of 82 to 87. Compared with non-SHS-exposed children, children with home SHS exposure had a lower mean HRQOL score for baseline health (adjusted mean difference –3.04 [95% CI -4.34, –1.74]). In analyses stratified by diagnosis, baseline health scores were lower for SHS-exposed children for all four conditions, but differences were statistically significant only for bronchiolitis (adjusted mean difference –2.94 [–5.0, –0.89]) and pneumonia (adjusted mean value –4.13 [–6.82, –1.44]; Table 2); none of these differences met the MCID threshold.

The overall mean HRQOL score at the time of admission was 56 (23), with a range across diagnoses of 49 to 61, with lower scores noted among SHS-exposed children compared with non-SHS-exposed children (adjusted mean difference –2.16 [–4.22, –0.10]). Similar to scores representing baseline health, admission scores were lower across all four conditions for SHS-exposed children. Only children with croup, however, had significantly lower admission scores that also met the MCID threshold (adjusted mean difference –5.71 [–10.67, –0.75]; Table 2).

To assess for potential dose-response effects of SHS exposure on HRQOL, we stratified SHS-exposed children into those with one smoker in the home (n = 513) and those with ≥2 smokers in the home (n = 215). Compared with non-SHS-exposed children, both HRQOL scores (baseline health and admission) were lower for SHS-exposed children. Consistent with a dose-response association, scores were lowest for children with ≥2 smokers in the home, both at baseline health (adjusted mean difference –3.92 [–6.03, –1.81]) and on admission (adjusted mean difference –3.67 [–6.98, –0.36]; Table 3).

Within a multicenter cohort of 2,309 children hospitalized with ARI, we noted significantly lower HRQOL scores among children exposed to SHS in the home as compared with nonexposed children. Differences were greatest for children living with ≥2 smokers in the home. In analyses stratified by diagnosis, differences in baseline health HRQOL scores were greatest for children with bronchiolitis and pneumonia. Differences in acute illness scores were greatest for children with croup.¹⁶

Our study provides evidence for acute and chronic impacts of SHS on HRQOL in children hospitalized with ARI. Although several studies have linked SHS exposure to reduced HRQOL in adults,^19,20 few similar studies have been conducted in children. Nonetheless, a wealth of studies have documented the negative impact of SHS exposure on clinical outcomes among children with ARI.^8,10,21-23 Our findings that home SHS exposure was associated with reduced HRQOL among our cohort of children with ARI are therefore consistent with related findings in adults and children. The observation that the effects of SHS exposure on HRQOL were greatest among children living with ≥2 smokers provides further evidence of a potential causal link between regular SHS exposure and HRQOL.

Although the magnitude and significance of associations between SHS exposure and HRQOL varied for each of the four diagnoses for baseline health and the acute illness, it is important to note that the point estimates for the adjusted mean differences were uniformly lower for the SHS-exposed children in each subgroup. Even so, only acute illness scores for croup exceeded the MCID threshold.¹⁶ Croup is the only included condition of the upper airway and is characterized by laryngotracheal inflammation leading to the typical cough and, in moderate to severe cases, stridor. Given that chronic SHS exposure induces a proinflammatory state,³ it is possible that SHS-exposed children with croup had more severe illness compared with nonexposed children with croup resulting in lower HRQOL scores on admission. Further, perceived differences in illness severity and HRQOL may be more readily apparent in children with croup (eg, stridor at rest vs intermittent or no stridor) as compared with children with lower respiratory tract diseases.

Of the four included diagnoses, the link between SHS exposure and asthma outcomes has been most studied. Prior work has demonstrated more frequent and severe acute exacerbations, as well as worse long-term lung function among SHS-exposed children as compared with nonexposed children.^22-24 It was, therefore, surprising that our study failed to demonstrate associations between SHS exposure and HRQOL among children with asthma. Reasons for this finding are unclear. One hypothesis is that caregivers of SHS-exposed children with asthma may be more aware of the impacts of SHS exposure on respiratory health (through prior education) and, thus, more likely to modify their smoking behaviors, or for their children to be on daily asthma controller therapy. Alternatively, caregivers of children with asthma may be more likely to underreport home SHS exposure. Thirty-eight percent of children with asthma, however, were classified as SHS-exposed. This percentage was greater than the other three conditions studied (25%-32%), suggesting that differential bias in underreporting was minimal. Given that children with asthma were older, on average, than children with the other three conditions, it may also be that these children spent more time in smoke-free environments (eg, school).

Nearly one-third of children in our study were exposed to SHS in the home. This is similar to the prevalence of exposure in other studies conducted among hospitalized children^8,10,21,25 but higher than the national prevalence of home SHS exposure among children in the United States.²⁶ Thus, hospitalized children represent a particularly vulnerable population and an important target for interventions aiming to reduce exposure to SHS. Although longitudinal interventions are likely necessary to affect long-term success, hospitalization for ARI may serve as a powerful teachable moment to begin cessation efforts. Hospitalization also offers time beyond a typical primary care outpatient encounter to focus on cessation counseling and may be the only opportunity to engage in counseling activities for some families with limited time or access. Further, prior studies have demonstrated both the feasibility and the effectiveness of smoking cessation interventions in hospitalized children.^27-30 Unfortunately, however, SHS exposure is often not documented at the time of hospitalization, and many opportunities to intervene are missed.^25,31 Thus, there is a need for improved strategies to reliably identify and intervene on SHS-exposed children in the hospital setting.

These findings should be considered in the context of several limitations. The observational nature of our study raises the potential for confounding, specifically with regard to socioeconomic status, as this is associated with both SHS exposure and lower HRQOL. Our modeling approach attempted to control for several factors associated with socioeconomic status, including caregiver education and insurance coverage, but there is potential for residual confounding. No single question is sufficient to fully assess SHS exposure as the intensity of home SHS exposure likely varies widely, and some children may be exposed to SHS outside of the home environment.³² The home, however, is often the most likely source of regular SHS exposure,^33,34 especially among young children (our cohort’s mean age was 3.6 years). Misclassification of SHS exposure is also possible due to underreporting of smoking.^35,36 As a result, some children regularly exposed to SHS may have been misclassified as nonexposed, and the observed associations between SHS exposure and HRQOL may be underestimated. Confirming our study’s findings using objective assessments of SHS exposure, such as cotinine, are warranted. Given the young age of our cohort, the PedsQL surveys were completed by the parent or legal guardian only in >90% of the enrolled subjects, and caregiver perceptions may not accurately reflect the child’s perceptions. Prior work, however, has demonstrated the validity of parent-proxy reporting of the PedsQL, including correlation with child self-report.³⁷ In our study, correlation between child and caregiver reporting (when available) was also very good (r = 0.72, 95% CI 0.64, 0.77). It is also possible that the timing of the HRQOL assessments (on admission) may have biased perceptions of baseline HRQOL, although we anticipate any bias would likely be nondifferential between SHS-exposed and nonexposed children and across diagnoses.

Nearly one-third of children in our study were exposed to SHS exposure in the home, and SHS exposure was associated with lower HRQOL for baseline health and during acute illness, providing further evidence of the dangers of SHS. Much work is needed in order to eliminate the impact of SHS on child health and families of children hospitalized for respiratory illness should be considered a priority population for smoking cessation efforts.

Acknowledgment

The authors wish to acknowledge the efforts of PRIS-PRIMES study team. The authors also wish to thank the children and families who consented to be a part of the PRIMES study.

Disclosures

The authors have no conflicts of interest relevant to this article to disclose.

Funding

This study was supported by NIH-NHLBI 1R01HL121067 to RMS.

Acute respiratory illnesses (ARIs), including acute exacerbations of asthma, croup, pneumonia, and bronchiolitis, are among the most common illnesses in childhood.¹ Although most ARIs can be managed in the outpatient setting, hospitalization is common with respiratory illnesses accounting for >425,000 hospitalizations annually.¹ Pneumonia, asthma, and bronchiolitis each rank among the top five reasons for pediatric hospitalization in the United States.¹ Successful efforts to prevent or mitigate the severity of ARIs could have a major impact on child health.

Exposure to secondhand smoke (SHS) is a preventable risk factor for ARI in children, particularly when there is regular exposure in the home.² Chronic exposure to SHS impacts systemic inflammation by suppressing serum interferon-gamma,³ which can lead to increased susceptibility to viral and bacterial infections,⁴ and increasing Th2 (atopic) cytokine expression, which is associated with asthma.⁵ SHS exposure in children has also been linked to diminished lung function.⁶ As a result, SHS exposure is associated with increased ARI susceptibility and severity in children.^7-10

Much research has focused on the clinical impact of SHS exposure on respiratory health in children, but little is known about the impact on patient-reported outcomes, such as health-related quality of life (HRQOL). Patient-reported outcomes help provide a comprehensive evaluation of the effectiveness of healthcare delivery systems. These outcomes are increasingly used by health service researchers to better understand patient and caregiver perspectives.¹¹ Given the known associations between SHS exposure and ARI morbidity, we postulated that regular SHS exposure would also impact HRQOL in children. In this study, we assessed the relationship between SHS exposure and HRQOL within a large, multicenter, prospective cohort of children presenting to the emergency department (ED) and/or hospital with ARI.

Study Population

This study was nested within the Pediatric Respiratory Illness Measurement System (PRIMES) study, a prospective cohort study of children with ARI in the ED and inpatient settings at five tertiary care children’s hospitals within the Pediatric Research in Inpatient Settings Network in Colorado, Pennsylvania, Tennessee, Texas, and Washington. Eligible children were two weeks to 16 years of age hospitalized after presenting to the ED with a primary diagnosis of asthma, croup, bronchiolitis, or pneumonia between July 1, 2014 and June 30, 2016. Because of an anticipated low frequency of croup hospitalizations, we also included children presenting to the ED and then discharged to home with this diagnosis. Children were assigned to a PRIMES diagnosis group based on their final discharge diagnosis. If there was a discrepancy between admission and discharge diagnoses, the discharge diagnosis was used. If a child had more than one discharge diagnosis for a PRIMES condition (eg, acute asthma and pneumonia), we chose the PRIMES condition with the lowest total enrollments overall. If the final discharge diagnosis was not a PRIMES condition, the case was excluded from further analysis. Patients with immunodeficiency, cystic fibrosis, a history of prematurity <32 weeks, chronic neuromuscular disease, cardiovascular disease, pulmonary diseases (other than asthma), and moderate to severe developmental delay were also excluded. Children admitted to intensive care were eligible only if they were transferred to an acute care ward <72 hours following admission. A survey was administered at the time of enrollment that collected information on SHS exposure, HRQOL, healthcare utilization, and demographics. All study procedures were reviewed and approved by the institutional review boards at each of the participating hospitals.

To ascertain SHS exposure, we asked caregivers, “How many persons living in the child’s home smoke?” Responses were dichotomized into non-SHS exposed (0 smokers) and SHS exposed (≥1 smokers). Children with missing data on home SHS exposure were excluded.

Health-Related Quality of Life Outcomes

We estimated HRQOL using the Pediatric Quality of Life (PedsQL^TM) 4.0 Generic Core and Infant Scales. The PedsQL instruments are validated, population HRQOL measures that evaluate the physical, mental, emotional, and social functioning of children two to 18 years old based on self- or caregiver-proxy report.^12-15 These instruments have also shown responsiveness as well as construct and predictive validity in hospitalized children.¹¹ For this study, we focused on the PedsQL physical functioning subscale, which assesses for problems with physical activities (eg, sports activity or exercise, low energy, and hurts or aches) on a five-point Likert scale (never to almost always a problem). Scores range from 0 to 100 with higher scores indicating a better HRQOL. The reported minimal clinically important difference (MCID), defined as the smallest difference in which individuals would perceive a benefit or would necessitate a change in management, for this scale is 4.5 points.^16,17

Children >8 years old were invited to complete the self-report version of the PedsQL. For children <8 years old, and for older children who were unable to complete them, surveys were completed by a parent or legal guardian. Respondents were asked to assess perceptions of their (or their child’s) HRQOL during periods of baseline health (the child’s usual state of health in the month preceding the current illness) and during the acute illness (the child’s state of health at the time of admission) as SHS exposure may influence perceptions of general health and/or contribute to worse outcomes during periods of acute illness.

Covariates collected at the time of enrollment included sociodemographics (child age, gender, race/ethnicity, and caregiver education), and healthcare utilization (caregiver-reported patient visits to a healthcare provider in the preceding six months). Insurance status and level of medical complexity (using the Pediatric Medical Complexity Algorithm)¹⁸ were obtained using the Pediatric Hospital Information System database, an administrative database containing clinical and resource utilization data from >45 children’s hospitals in the United States including all of the PRIMES study hospitals.¹³

Analysis

Descriptive statistics included frequency (%) and mean (standard deviation). Bivariate comparisons according to SHS exposure status were analyzed using chi-squared tests for categorical variables and analysis of variance for continuous variables. Multivariable linear mixed regression models were used to examine associations between home SHS exposure and HRQOL for baseline health and during admission, overall and stratified by diagnosis. Covariates in each model included age, sex, race/ethnicity, caregiver education, and healthcare visits in the preceding six months. We also included a hospital random effect to account for clustering of patients within hospitals and used robust standard errors for inference.

In a secondary analysis to explore potential dose-response effects of SHS exposure, we examined associations between an ordinal exposure variable (0 smokers, 1 smoker, ≥2 smokers) and HRQOL for baseline health and during admission for the acute illness. Because of sample size limitations, diagnosis-specific analyses examining dose-response effects were not conducted.

Study Population

Of the 2,334 children enrolled in the PRIMES study, 25 (1%) respondents did not report on home SHS exposure and were excluded, yielding a final study population of 2,309 children, of whom 728 (32%) had reported home SHS exposure. The study population included 664 children with asthma (mean age seven years [3.5]; 38% with home SHS exposure), 740 with bronchiolitis (mean age 0.7 years [0.5]; 32% with home SHS exposure), 342 with croup (mean age 1.7 [1.1]; 25% with home SHS exposure), and 563 with pneumonia (mean age 4.4 [3.8]; 27% with home SHS exposure; Table 1). Compared with non-SHS-exposed children, those with home SHS exposure tend to be slightly older (3.9 vs 3.4 years, P = .01), more likely to be non-Hispanic Black (29% vs 19%, P < .001), to have a chronic condition (52% vs 41%, P < .001), to come from a household where caregiver(s) did not graduate from college (45% vs 29%, P < .001), and to have public insurance (73% vs 49%, P < .001).

Home SHS Exposure and Health-related Quality of Life

The overall mean HRQOL score for baseline health was 83 (15), with a range across diagnoses of 82 to 87. Compared with non-SHS-exposed children, children with home SHS exposure had a lower mean HRQOL score for baseline health (adjusted mean difference –3.04 [95% CI -4.34, –1.74]). In analyses stratified by diagnosis, baseline health scores were lower for SHS-exposed children for all four conditions, but differences were statistically significant only for bronchiolitis (adjusted mean difference –2.94 [–5.0, –0.89]) and pneumonia (adjusted mean value –4.13 [–6.82, –1.44]; Table 2); none of these differences met the MCID threshold.

The overall mean HRQOL score at the time of admission was 56 (23), with a range across diagnoses of 49 to 61, with lower scores noted among SHS-exposed children compared with non-SHS-exposed children (adjusted mean difference –2.16 [–4.22, –0.10]). Similar to scores representing baseline health, admission scores were lower across all four conditions for SHS-exposed children. Only children with croup, however, had significantly lower admission scores that also met the MCID threshold (adjusted mean difference –5.71 [–10.67, –0.75]; Table 2).

To assess for potential dose-response effects of SHS exposure on HRQOL, we stratified SHS-exposed children into those with one smoker in the home (n = 513) and those with ≥2 smokers in the home (n = 215). Compared with non-SHS-exposed children, both HRQOL scores (baseline health and admission) were lower for SHS-exposed children. Consistent with a dose-response association, scores were lowest for children with ≥2 smokers in the home, both at baseline health (adjusted mean difference –3.92 [–6.03, –1.81]) and on admission (adjusted mean difference –3.67 [–6.98, –0.36]; Table 3).

Within a multicenter cohort of 2,309 children hospitalized with ARI, we noted significantly lower HRQOL scores among children exposed to SHS in the home as compared with nonexposed children. Differences were greatest for children living with ≥2 smokers in the home. In analyses stratified by diagnosis, differences in baseline health HRQOL scores were greatest for children with bronchiolitis and pneumonia. Differences in acute illness scores were greatest for children with croup.¹⁶

Our study provides evidence for acute and chronic impacts of SHS on HRQOL in children hospitalized with ARI. Although several studies have linked SHS exposure to reduced HRQOL in adults,^19,20 few similar studies have been conducted in children. Nonetheless, a wealth of studies have documented the negative impact of SHS exposure on clinical outcomes among children with ARI.^8,10,21-23 Our findings that home SHS exposure was associated with reduced HRQOL among our cohort of children with ARI are therefore consistent with related findings in adults and children. The observation that the effects of SHS exposure on HRQOL were greatest among children living with ≥2 smokers provides further evidence of a potential causal link between regular SHS exposure and HRQOL.

Although the magnitude and significance of associations between SHS exposure and HRQOL varied for each of the four diagnoses for baseline health and the acute illness, it is important to note that the point estimates for the adjusted mean differences were uniformly lower for the SHS-exposed children in each subgroup. Even so, only acute illness scores for croup exceeded the MCID threshold.¹⁶ Croup is the only included condition of the upper airway and is characterized by laryngotracheal inflammation leading to the typical cough and, in moderate to severe cases, stridor. Given that chronic SHS exposure induces a proinflammatory state,³ it is possible that SHS-exposed children with croup had more severe illness compared with nonexposed children with croup resulting in lower HRQOL scores on admission. Further, perceived differences in illness severity and HRQOL may be more readily apparent in children with croup (eg, stridor at rest vs intermittent or no stridor) as compared with children with lower respiratory tract diseases.

Of the four included diagnoses, the link between SHS exposure and asthma outcomes has been most studied. Prior work has demonstrated more frequent and severe acute exacerbations, as well as worse long-term lung function among SHS-exposed children as compared with nonexposed children.^22-24 It was, therefore, surprising that our study failed to demonstrate associations between SHS exposure and HRQOL among children with asthma. Reasons for this finding are unclear. One hypothesis is that caregivers of SHS-exposed children with asthma may be more aware of the impacts of SHS exposure on respiratory health (through prior education) and, thus, more likely to modify their smoking behaviors, or for their children to be on daily asthma controller therapy. Alternatively, caregivers of children with asthma may be more likely to underreport home SHS exposure. Thirty-eight percent of children with asthma, however, were classified as SHS-exposed. This percentage was greater than the other three conditions studied (25%-32%), suggesting that differential bias in underreporting was minimal. Given that children with asthma were older, on average, than children with the other three conditions, it may also be that these children spent more time in smoke-free environments (eg, school).

Nearly one-third of children in our study were exposed to SHS in the home. This is similar to the prevalence of exposure in other studies conducted among hospitalized children^8,10,21,25 but higher than the national prevalence of home SHS exposure among children in the United States.²⁶ Thus, hospitalized children represent a particularly vulnerable population and an important target for interventions aiming to reduce exposure to SHS. Although longitudinal interventions are likely necessary to affect long-term success, hospitalization for ARI may serve as a powerful teachable moment to begin cessation efforts. Hospitalization also offers time beyond a typical primary care outpatient encounter to focus on cessation counseling and may be the only opportunity to engage in counseling activities for some families with limited time or access. Further, prior studies have demonstrated both the feasibility and the effectiveness of smoking cessation interventions in hospitalized children.^27-30 Unfortunately, however, SHS exposure is often not documented at the time of hospitalization, and many opportunities to intervene are missed.^25,31 Thus, there is a need for improved strategies to reliably identify and intervene on SHS-exposed children in the hospital setting.

These findings should be considered in the context of several limitations. The observational nature of our study raises the potential for confounding, specifically with regard to socioeconomic status, as this is associated with both SHS exposure and lower HRQOL. Our modeling approach attempted to control for several factors associated with socioeconomic status, including caregiver education and insurance coverage, but there is potential for residual confounding. No single question is sufficient to fully assess SHS exposure as the intensity of home SHS exposure likely varies widely, and some children may be exposed to SHS outside of the home environment.³² The home, however, is often the most likely source of regular SHS exposure,^33,34 especially among young children (our cohort’s mean age was 3.6 years). Misclassification of SHS exposure is also possible due to underreporting of smoking.^35,36 As a result, some children regularly exposed to SHS may have been misclassified as nonexposed, and the observed associations between SHS exposure and HRQOL may be underestimated. Confirming our study’s findings using objective assessments of SHS exposure, such as cotinine, are warranted. Given the young age of our cohort, the PedsQL surveys were completed by the parent or legal guardian only in >90% of the enrolled subjects, and caregiver perceptions may not accurately reflect the child’s perceptions. Prior work, however, has demonstrated the validity of parent-proxy reporting of the PedsQL, including correlation with child self-report.³⁷ In our study, correlation between child and caregiver reporting (when available) was also very good (r = 0.72, 95% CI 0.64, 0.77). It is also possible that the timing of the HRQOL assessments (on admission) may have biased perceptions of baseline HRQOL, although we anticipate any bias would likely be nondifferential between SHS-exposed and nonexposed children and across diagnoses.

Nearly one-third of children in our study were exposed to SHS exposure in the home, and SHS exposure was associated with lower HRQOL for baseline health and during acute illness, providing further evidence of the dangers of SHS. Much work is needed in order to eliminate the impact of SHS on child health and families of children hospitalized for respiratory illness should be considered a priority population for smoking cessation efforts.

Acknowledgment

The authors wish to acknowledge the efforts of PRIS-PRIMES study team. The authors also wish to thank the children and families who consented to be a part of the PRIMES study.

Disclosures

The authors have no conflicts of interest relevant to this article to disclose.

Funding

This study was supported by NIH-NHLBI 1R01HL121067 to RMS.

Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.¹ Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event² when it does occur.

In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations^3-5 and may feel unprepared to handle crisis events such as cardiac arrest.⁶ Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.

Survey Development

We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).^7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,¹⁰ and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.

Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.

Participants and Data Collection

All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.

Data Analysis

Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.

We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.

We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.

A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).

Assessment of Exposure

Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.

There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).

Assessment of Confidence

Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.

Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).

We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.

To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.¹¹

Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.

An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.¹² In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.

Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.¹³ Tools such as our end-of-year assessment offer an alternative method that limits these barriers.

Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.

Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties^14-24 and in nonmedical domains such as aviation.^25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events^28-30 and may even be superior to traditional clinical medical education.³¹ In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations^32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.

Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,⁷ likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.^34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.

We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.

Disclosures

The authors declare no conflict of interest.

Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.¹ Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event² when it does occur.

In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations^3-5 and may feel unprepared to handle crisis events such as cardiac arrest.⁶ Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.

Survey Development

We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).^7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,¹⁰ and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.

Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.

Participants and Data Collection

All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.

Data Analysis

Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.

We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.

We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.

A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).

Assessment of Exposure

Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.

There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).

Assessment of Confidence

Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.

Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).

We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.

To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.¹¹

Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.

An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.¹² In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.

Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.¹³ Tools such as our end-of-year assessment offer an alternative method that limits these barriers.

Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.

Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties^14-24 and in nonmedical domains such as aviation.^25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events^28-30 and may even be superior to traditional clinical medical education.³¹ In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations^32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.

Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,⁷ likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.^34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.

We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.

Disclosures

The authors declare no conflict of interest.

Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.¹ Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event² when it does occur.

In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations^3-5 and may feel unprepared to handle crisis events such as cardiac arrest.⁶ Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.

Survey Development

We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).^7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,¹⁰ and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.

Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.

Participants and Data Collection

All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.

Data Analysis

Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.

We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.

We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.

A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).

Assessment of Exposure

Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.

There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).

Assessment of Confidence

Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.

Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).

We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.

To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.¹¹

Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.

An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.¹² In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.

Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.¹³ Tools such as our end-of-year assessment offer an alternative method that limits these barriers.

Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.

Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties^14-24 and in nonmedical domains such as aviation.^25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events^28-30 and may even be superior to traditional clinical medical education.³¹ In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations^32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.

Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,⁷ likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.^34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.

We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.

Disclosures

The authors declare no conflict of interest.

Graduate medical education (GME) is heavily reliant on experiential learning. Most of a resident’s time is spent in progressively independent delivery of patient care, which is associated with decreasing supervision. Attainment and demonstration of competence in patient care is the goal and responsibility of GME training programs. What happens, then, if the medicine resident never has the experience necessary to enable experiential learning? What if she never “sees one,” let alone “does one”?

In this month’s Journal of Hospital Medicine, Sclafani et al¹ examine how exposure to urgent clinical situations impacts residents’ confidence in managing these ward emergencies. They astutely reveal the idiosyncratic nature of residency training and consequent gaps created when an educational delivery model predicated on experience lacks certain experiences. How can a resident without certain key experiences be ready for independent practice?

The ACGME’s Next Accreditation System is intended to ensure that residents are prepared for independent practice. The educational outcomes that learners must attain are comprised of six core competencies, with milestones intended to operationalize the measurement and reporting of learner progression toward competence.^2,3 It is challenging to apply general competencies to assessment of day to day clinical activities. This challenge led to the development of 16 Entrustable Professional Activities (EPAs). These allow the direct observation of concrete clinical activities that could then infer the attainment (or not) of multiple competencies. Ideally, EPAs are paired with and mapped to curricular milestones which describe a learner’s trajectory within the framework of competencies and determine if a resident is prepared for independent practice.^4,5

In Sclafani et al.¹ the authors characterize resident exposure to, and confidence in, 50 urgent clinical situations. Both level of training and exposure were associated with increased confidence. However, the most important finding of this paper is the wide variation of resident exposures and confidence with respect to specific urgent clinical events. At least 15% of graduating residents had never seen 16% of the 50 emergency events, and a majority of graduating residents did not feel confident managing 20% of the 50 events, highlighting the idiosyncratic nature of GME training.¹ Of course, while certain entities on the list of clinical emergencies were not identified as final diagnoses, it is possible they were still considered in the process of caring for patients in different situations.

Several factors account for the idiosyncratic nature of medical training, including the rarity of certain clinical events, seasonal variation in conditions, and other variables (ie, learner elective choices). In addition, the scheduling of most residency programs is based on patient care needs instead of individual trainees’ educational needs. Other areas of medicine have attempted to standardize experience and ensure specific exposure and/or competence using strategies such as surgical case logs and case-based certifying examinations. There are very important recently described projects in undergraduate medical education aimed at using longitudinal assessment of EPAs in multiple contexts to make entrustment decisions.⁶ However, Internal Medicine residencies do not routinely employ these strategies.

It must be noted that Sclafani et al. surveyed residents from only one site, and examined only self-reported exposure and confidence, not competence. The relationship between confidence and competence is notoriously problematic⁷ and there is a risk of familiarity creating an illusion of knowledge and/or competence. Ultimately, a competency-based medical system is intended to be dynamic, adaptive, and contextual. Despite the extensive competency-based framework in place to track the development of physicians, data about the contexts in which competency is demonstrated are lacking. There is no reason to think that the key gaps identified in Sclafani et al are unique to their institution.

Given the ultimate goal of developing curricula that prepare residents for independent practice coupled with robust systems of assessment that ensure they are ready to do so, educators must implement strategies to identify and alleviate the idiosyncrasy of the resident experience. The survey tool in the present work could be used as a needs assessment and would require minimal resources, but is limited by recall bias, illusion of knowledge, and lack of data regarding actual competence. Other potential strategies include case logs or e-folios, although these tools are also limited by the understanding that familiarity and exposure do not necessarily engender competence.

One potential strategy suggested by Warm et al. is the addition of the “Observable Practice Activities” (OPA), “a collection of learning objectives/activities that must be observed in daily practice in order to form entrustment decisions.”⁸ The intention is to more granularly define what residents actually do and then map these activities to the established competency-based framework. Using these observable activities as an assessment unit may allow for identification of individual experience gaps, thereby improving the dynamicity and adaptiveness of GME training. Certainly, there are very real concerns about further complicating an already complex and abstract system and using a reductionist approach to define the activities of a profession. However, the findings of Sclafani et al with respect to the wide range of resident experience elucidates the need for continued study and innovation regarding the manner in which the medical education community determines our trainees are prepared for independent practice.

Disclosures

The authors have nothing to disclose.

Graduate medical education (GME) is heavily reliant on experiential learning. Most of a resident’s time is spent in progressively independent delivery of patient care, which is associated with decreasing supervision. Attainment and demonstration of competence in patient care is the goal and responsibility of GME training programs. What happens, then, if the medicine resident never has the experience necessary to enable experiential learning? What if she never “sees one,” let alone “does one”?

In this month’s Journal of Hospital Medicine, Sclafani et al¹ examine how exposure to urgent clinical situations impacts residents’ confidence in managing these ward emergencies. They astutely reveal the idiosyncratic nature of residency training and consequent gaps created when an educational delivery model predicated on experience lacks certain experiences. How can a resident without certain key experiences be ready for independent practice?

The ACGME’s Next Accreditation System is intended to ensure that residents are prepared for independent practice. The educational outcomes that learners must attain are comprised of six core competencies, with milestones intended to operationalize the measurement and reporting of learner progression toward competence.^2,3 It is challenging to apply general competencies to assessment of day to day clinical activities. This challenge led to the development of 16 Entrustable Professional Activities (EPAs). These allow the direct observation of concrete clinical activities that could then infer the attainment (or not) of multiple competencies. Ideally, EPAs are paired with and mapped to curricular milestones which describe a learner’s trajectory within the framework of competencies and determine if a resident is prepared for independent practice.^4,5

In Sclafani et al.¹ the authors characterize resident exposure to, and confidence in, 50 urgent clinical situations. Both level of training and exposure were associated with increased confidence. However, the most important finding of this paper is the wide variation of resident exposures and confidence with respect to specific urgent clinical events. At least 15% of graduating residents had never seen 16% of the 50 emergency events, and a majority of graduating residents did not feel confident managing 20% of the 50 events, highlighting the idiosyncratic nature of GME training.¹ Of course, while certain entities on the list of clinical emergencies were not identified as final diagnoses, it is possible they were still considered in the process of caring for patients in different situations.

Several factors account for the idiosyncratic nature of medical training, including the rarity of certain clinical events, seasonal variation in conditions, and other variables (ie, learner elective choices). In addition, the scheduling of most residency programs is based on patient care needs instead of individual trainees’ educational needs. Other areas of medicine have attempted to standardize experience and ensure specific exposure and/or competence using strategies such as surgical case logs and case-based certifying examinations. There are very important recently described projects in undergraduate medical education aimed at using longitudinal assessment of EPAs in multiple contexts to make entrustment decisions.⁶ However, Internal Medicine residencies do not routinely employ these strategies.

It must be noted that Sclafani et al. surveyed residents from only one site, and examined only self-reported exposure and confidence, not competence. The relationship between confidence and competence is notoriously problematic⁷ and there is a risk of familiarity creating an illusion of knowledge and/or competence. Ultimately, a competency-based medical system is intended to be dynamic, adaptive, and contextual. Despite the extensive competency-based framework in place to track the development of physicians, data about the contexts in which competency is demonstrated are lacking. There is no reason to think that the key gaps identified in Sclafani et al are unique to their institution.

Given the ultimate goal of developing curricula that prepare residents for independent practice coupled with robust systems of assessment that ensure they are ready to do so, educators must implement strategies to identify and alleviate the idiosyncrasy of the resident experience. The survey tool in the present work could be used as a needs assessment and would require minimal resources, but is limited by recall bias, illusion of knowledge, and lack of data regarding actual competence. Other potential strategies include case logs or e-folios, although these tools are also limited by the understanding that familiarity and exposure do not necessarily engender competence.

One potential strategy suggested by Warm et al. is the addition of the “Observable Practice Activities” (OPA), “a collection of learning objectives/activities that must be observed in daily practice in order to form entrustment decisions.”⁸ The intention is to more granularly define what residents actually do and then map these activities to the established competency-based framework. Using these observable activities as an assessment unit may allow for identification of individual experience gaps, thereby improving the dynamicity and adaptiveness of GME training. Certainly, there are very real concerns about further complicating an already complex and abstract system and using a reductionist approach to define the activities of a profession. However, the findings of Sclafani et al with respect to the wide range of resident experience elucidates the need for continued study and innovation regarding the manner in which the medical education community determines our trainees are prepared for independent practice.

Disclosures

The authors have nothing to disclose.

Graduate medical education (GME) is heavily reliant on experiential learning. Most of a resident’s time is spent in progressively independent delivery of patient care, which is associated with decreasing supervision. Attainment and demonstration of competence in patient care is the goal and responsibility of GME training programs. What happens, then, if the medicine resident never has the experience necessary to enable experiential learning? What if she never “sees one,” let alone “does one”?

In this month’s Journal of Hospital Medicine, Sclafani et al¹ examine how exposure to urgent clinical situations impacts residents’ confidence in managing these ward emergencies. They astutely reveal the idiosyncratic nature of residency training and consequent gaps created when an educational delivery model predicated on experience lacks certain experiences. How can a resident without certain key experiences be ready for independent practice?

The ACGME’s Next Accreditation System is intended to ensure that residents are prepared for independent practice. The educational outcomes that learners must attain are comprised of six core competencies, with milestones intended to operationalize the measurement and reporting of learner progression toward competence.^2,3 It is challenging to apply general competencies to assessment of day to day clinical activities. This challenge led to the development of 16 Entrustable Professional Activities (EPAs). These allow the direct observation of concrete clinical activities that could then infer the attainment (or not) of multiple competencies. Ideally, EPAs are paired with and mapped to curricular milestones which describe a learner’s trajectory within the framework of competencies and determine if a resident is prepared for independent practice.^4,5

In Sclafani et al.¹ the authors characterize resident exposure to, and confidence in, 50 urgent clinical situations. Both level of training and exposure were associated with increased confidence. However, the most important finding of this paper is the wide variation of resident exposures and confidence with respect to specific urgent clinical events. At least 15% of graduating residents had never seen 16% of the 50 emergency events, and a majority of graduating residents did not feel confident managing 20% of the 50 events, highlighting the idiosyncratic nature of GME training.¹ Of course, while certain entities on the list of clinical emergencies were not identified as final diagnoses, it is possible they were still considered in the process of caring for patients in different situations.

Several factors account for the idiosyncratic nature of medical training, including the rarity of certain clinical events, seasonal variation in conditions, and other variables (ie, learner elective choices). In addition, the scheduling of most residency programs is based on patient care needs instead of individual trainees’ educational needs. Other areas of medicine have attempted to standardize experience and ensure specific exposure and/or competence using strategies such as surgical case logs and case-based certifying examinations. There are very important recently described projects in undergraduate medical education aimed at using longitudinal assessment of EPAs in multiple contexts to make entrustment decisions.⁶ However, Internal Medicine residencies do not routinely employ these strategies.

It must be noted that Sclafani et al. surveyed residents from only one site, and examined only self-reported exposure and confidence, not competence. The relationship between confidence and competence is notoriously problematic⁷ and there is a risk of familiarity creating an illusion of knowledge and/or competence. Ultimately, a competency-based medical system is intended to be dynamic, adaptive, and contextual. Despite the extensive competency-based framework in place to track the development of physicians, data about the contexts in which competency is demonstrated are lacking. There is no reason to think that the key gaps identified in Sclafani et al are unique to their institution.

Given the ultimate goal of developing curricula that prepare residents for independent practice coupled with robust systems of assessment that ensure they are ready to do so, educators must implement strategies to identify and alleviate the idiosyncrasy of the resident experience. The survey tool in the present work could be used as a needs assessment and would require minimal resources, but is limited by recall bias, illusion of knowledge, and lack of data regarding actual competence. Other potential strategies include case logs or e-folios, although these tools are also limited by the understanding that familiarity and exposure do not necessarily engender competence.

One potential strategy suggested by Warm et al. is the addition of the “Observable Practice Activities” (OPA), “a collection of learning objectives/activities that must be observed in daily practice in order to form entrustment decisions.”⁸ The intention is to more granularly define what residents actually do and then map these activities to the established competency-based framework. Using these observable activities as an assessment unit may allow for identification of individual experience gaps, thereby improving the dynamicity and adaptiveness of GME training. Certainly, there are very real concerns about further complicating an already complex and abstract system and using a reductionist approach to define the activities of a profession. However, the findings of Sclafani et al with respect to the wide range of resident experience elucidates the need for continued study and innovation regarding the manner in which the medical education community determines our trainees are prepared for independent practice.

Disclosures

The authors have nothing to disclose.

Hip fracture is a common reason for unexpected, urgent inpatient surgery in older patients. In 2005, the incidence of hip fracture was 369.0 and 793.5 per 100,000 in men and women respectively.¹ These numbers declined over the preceding decade, potentially as a result of bisphosphonate use. Age- and risk-adjusted 30-day mortality rates for men and women in 2005 were approximately 10% and 5%, respectively.

Evidence suggests that timely surgical repair of hip fractures improves outcomes, although the optimal timing is controversial. Guidelines from the American College of Surgeons Committee on Trauma from 2015 recommend surgical intervention within 48 hours for geriatric hip fracures.² A 2008 systematic review found that operative delay beyond 48 hours was associated with a 41% increase in 30-day all-cause mortality and a 32% increase in one-year all-cause mortality.³ Recent evidence suggests that the rate of complications begins to increase with delays beyond 24 hours.⁴

There has been a focus over the past decade on overuse of preoperative testing for low- and intermediate-risk surgeries.^5-7 Beginning in 2012, the American Board of Internal Medicine initiated the Choosing Wisely^® campaign in which numerous societies issued recommendations on reducing utilization of various diagnostic tests, a number of which have focused on preoperative tests. Two groups—the American Society of Anesthesiologists (ASA) and the American Society of Echocardiography (ASE)— issued specific recommendations on preoperative cardiac testing.⁸ In February 2013, the ASE recommended avoiding preoperative echocardiograms in patients without a history or symptoms of heart disease. In October 2013, the ASA recommended against transthoracic echocardiogram (TTE), transesophageal echocardiogram (TEE), or stress testing for low- or intermediate-risk noncardiac surgery for patients with stable cardiac disease.

Finally, in 2014, the American College of Cardiology (ACC)/American Heart Association (AHA) issued updated perioperative guidelines for patients undergoing noncardiac surgeries.⁹ They recommended preoperative stress testing only in a small subset of cases (patients with an elevated perioperative risk of major adverse cardiac event, a poor or unknown functional capacity, or those in whom stress testing would impact perioperative care).

Given the high cost of preoperative cardiac testing, the potential for delays in care that can adversely impact outcomes, and the recent recommendations, we sought to characterize the rates of inpatient preoperative cardiac testing prior to hip fracture surgery in recent years and to see whether recent recommendations to curb use of these tests were temporally associated with changing rates.

Overview

We utilized two datasets—the Healthcare Cost and Utilization Project (HCUP) State Inpatient Databases (SID) and the American Hospital Association (AHA) Annual Survey—to characterize preoperative cardiac testing. SID data from Maryland, New Jersey, and Washington State from 2011 through September 2015 were used (the ICD coding system changed from ICD9 to ICD10 on October 1). This was combined with AHA data for these years. We included all hospitalizations with a primary ICD9 procedure code for hip fracture repair—78.55, 78.65, 79.05, 79.15, 79.25, 79.35, 79.45, 79.55, 79.65, 79.75, 79.85, and 79.95. We excluded all observations that involved an interhospital transfer. This study was exempt from institutional review board approval.

Measurement and Outcomes

We summarized demographic data for the hospitalizations that met the inclusion criteria as well as the associated hospitals. The primary outcome was the percentage of patients undergoing TTE, stress test, and cardiac catheterization during a hospitalization with a primary procedure code of hip fracture repair. Random effects logistic regression models for each type of diagnostic test were developed to determine the factors that might impact test utilization. In addition to running each test as a separate model, we also performed an analysis in which the outcome was performance of any of these three cardiac tests. Random effects were used to account for clustering of testing within hospitals. Variables included time (3-month intervals), state, age (continuous variable), gender, length of stay, payer (Medicare/Medicaid/private insurance/self-pay/other), hospital teaching status (major teaching/minor teaching/nonteaching), hospital size according to number of beds (continuous variable), and mortality score. Major teaching hospitals are defined as members of the Council of Teaching Hospitals. Minor teaching hospitals are defined as (1) those with one or more postgraduate training programs recognized by the American Council on Graduate Medical Education, (2) those with a medical school affiliation reported to the American Medical Association, or (3) those with an internship or residency approved by the American Osteopathic Association.

The SID has a specific binary indicator variable for each of the three diagnostic tests we evaluated. The use of the diagnostic test is evaluated through both UB-92 revenue codes and ICD9 procedure codes, with the presence of either leading to the indicator variable being positive.¹⁰ Finally, we performed a sensitivity analysis to evaluate the significance of changing utilization trends by interrupted time series analysis. A level of 0.05 was used to determine statistical significance. Analyses were done in STATA 15 (College Station, Texas).

The dataset included 75,144 hospitalizations with a primary procedure code of hip fracture over the study period (Table). The number of hospitalizations per year was fairly consistent over the study period in each state, although there were fewer hospitalizations for 2015 as this included only January through September. The mean age was 72.8 years, and 67% were female. The primary payer was Medicare for 71.7% of hospitalizations. Hospitalizations occurred at 181 hospitals, the plurality of which (42.9%) were minor teaching hospitals. The proportions of hospitalizations that included a TTE, stress test, and cardiac catheterization were 12.6%, 1.1%, and 0.5%, respectively. Overall, 13.5% of patients underwent any cardiac testing.

There was a statistically significantly lower rate of stress tests (odds ratio [OR], 0.32; 95% CI, 0.19-0.54) and cardiac catheterizations (OR, 0.46; 95% CI, 0.27-0.79) in Washington than in Maryland and New Jersey. Female gender was associated with significantly lower adjusted ORs for stress tests (OR, 0.74; 95% CI, 0.63-0.86) and cardiac catheterizations (OR, 0.73; 95% CI, 0.59-0.91), and increasing age was associated with higher adjusted ORs for each test (TTE, OR, 1.033; 95% CI, 1.031-1.035; stress tests, OR, 1.007; 95% CI, 1.001-1.013; cardiac catheterizations, OR, 1.011; 95% CI, 1.003-1.019). Private insurance was associated with a lower likelihood of stress tests (OR, 0.65; 95% CI, 0.50-0.85) and cardiac catheterizations (OR, 0.67; 95% CI,0.46-0.98), and self-pay was associated with a lower likelihood of TTE (OR, 0.76; 95% CI, 0.61-0.95) and stress test (OR, 0.43; 95% CI, 0.21-0.90), all compared with Medicare.

Larger hospitals were associated with a greater likelihood of cardiac catheterizations (OR, 1.18; 95% CI, 1.03-1.36) and a lower likelihood of TTE (OR, 0.89; 95% CI, 0.82-0.96). An unweighted average of these tests between 2011 and October 2015 showed a modest increase in TTEs and a modest decrease in stress tests and cardiac catheterizations (Figure). A multivariable random effects regression for use of TTEs revealed a significantly increasing trend from 2011 to 2014 (OR, 1.04, P < .0001), but the decreasing trend for 2015 was not statistically significant when analyzed according to quarters or months (for which data from only New Jersey and Washington are available).

There has been limited research into how often preoperative cardiac testing occurs in the inpatient setting. Our aim was to study its prevalence prior to hip fracture surgery during a time period when multiple recommendations had been issued to limit its use. We found rates of ischemic testing (stress tests and cardiac catheterizations) to be appropriately, and perhaps surprisingly, low. Our results on ischemic testing rates are consistent with previous studies, which have focused on the outpatient setting where much of the preoperative workup for nonurgent surgeries occurs. The rate of TTEs was higher than in previous studies of the outpatient preoperative setting, although it is unclear what an optimal rate of TTEs is.

A recent study examining outpatient preoperative stress tests within the 30 days before cataract surgeries, knee arthroscopies, or shoulder arthroscopies found a rate of 2.1% for Medicare fee-for-service patients in 2009 with little regional variation.¹¹ Another evaluation using 2009 Medicare claims data found rates of preoperative TTEs and stress tests to be 0.8% and 0.7%, respectively.¹² They included TTEs and stress tests performed within 30 days of a low- or intermediate-risk surgery. A study analyzing the rate of preoperative TTEs between 2009 and 2014 found that rates varied from 2.0% to 3.4% for commercially insured patients aged 50-64 years and Medicare-advantage patients, respectively, in 2009.¹³ These rates decreased by 7.0% and 12.6% from 2009 to 2014. These studies, like ours, suggest that preoperative cardiac testing has not been a major source of wasteful spending. One explanation for the higher rate of TTEs we observed in the inpatient setting might be that primary care physicians in the outpatient setting are more likely to have historical cardiac testing results compared with physicians in a hospital.

We found that the rate of stress testing and cardiac catheterization in Washington was significantly lower than that in Maryland and New Jersey. This is consistent with a number of measures of healthcare utilization – total Medicare reimbursement in the last six months of life, mean number of hospital days in the last six months of life, and healthcare intensity index—for all of which Washington was below the national mean and Maryland and New Jersey were above it.¹⁴

Finally, we found evidence of a lower rate of preoperative stress tests and cardiac catheterizations for women despite controlling for age and mortality score. Of course, we did not control directly for cardiovascular comorbidities; as a result, there could be residual confounding. However, these results are consistent with previous findings of gender bias in both pharmacologic management of coronary artery disease (CAD)¹⁵ and diagnostic testing for suspected CAD.¹⁶

We focused on hospitalizations with a primary procedure code to surgically treat hip fracture. We are unable to tell if the cardiac testing of these patients had occurred before or after the procedure. However, we suspect that the vast majority were completed for preoperative evaluation. It is likely that a small subset were done to diagnose and manage cardiac complications that either accompanied the hip fracture or occurred postoperatively. Another limitation is that we cannot determine if a patient had one of these tests recently in the emergency department or as an outpatient.

We also chose to include only patients who actually had hip fracture surgery. It is possible that the testing rate is higher for all patients admitted for hip fracture and that some of these patients did not have surgery because of abnormal cardiac testing. However, we suspect that this is a very small fraction given the high degree of morbidity and mortality associated with untreated hip fracture.

We found a low rate of preoperative cardiac testing in patients hospitalized for hip fracture surgery both in the years before and after the issuance of recommendations intended to curb its use. Although it is reassuring that the volume of low-value testing is lower than we expected, these findings highlight the importance of targeting utilization improvement efforts toward low-value tests and procedures that are more heavily used, since further curbing the use of infrequently utilized tests and procedures will have only a modest impact on overall healthcare expenditure. Our findings highlight the necessity that professional organizations ensure that they focus on true areas of inappropriate utilization. These are the areas in which improvements will have a major impact on healthcare spending. Further research should aim to quantify unwarranted cardiac testing for other inpatient surgeries that are less urgent, as the urgency of hip fracture repair may be driving the relatively low utilization of inpatient cardiac testing.

Disclosures

The authors have nothing to disclose.

Funding

This project was supported by the Johns Hopkins Hospitalist Scholars Fund and the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core.

Hip fracture is a common reason for unexpected, urgent inpatient surgery in older patients. In 2005, the incidence of hip fracture was 369.0 and 793.5 per 100,000 in men and women respectively.¹ These numbers declined over the preceding decade, potentially as a result of bisphosphonate use. Age- and risk-adjusted 30-day mortality rates for men and women in 2005 were approximately 10% and 5%, respectively.

Evidence suggests that timely surgical repair of hip fractures improves outcomes, although the optimal timing is controversial. Guidelines from the American College of Surgeons Committee on Trauma from 2015 recommend surgical intervention within 48 hours for geriatric hip fracures.² A 2008 systematic review found that operative delay beyond 48 hours was associated with a 41% increase in 30-day all-cause mortality and a 32% increase in one-year all-cause mortality.³ Recent evidence suggests that the rate of complications begins to increase with delays beyond 24 hours.⁴

There has been a focus over the past decade on overuse of preoperative testing for low- and intermediate-risk surgeries.^5-7 Beginning in 2012, the American Board of Internal Medicine initiated the Choosing Wisely^® campaign in which numerous societies issued recommendations on reducing utilization of various diagnostic tests, a number of which have focused on preoperative tests. Two groups—the American Society of Anesthesiologists (ASA) and the American Society of Echocardiography (ASE)— issued specific recommendations on preoperative cardiac testing.⁸ In February 2013, the ASE recommended avoiding preoperative echocardiograms in patients without a history or symptoms of heart disease. In October 2013, the ASA recommended against transthoracic echocardiogram (TTE), transesophageal echocardiogram (TEE), or stress testing for low- or intermediate-risk noncardiac surgery for patients with stable cardiac disease.

Finally, in 2014, the American College of Cardiology (ACC)/American Heart Association (AHA) issued updated perioperative guidelines for patients undergoing noncardiac surgeries.⁹ They recommended preoperative stress testing only in a small subset of cases (patients with an elevated perioperative risk of major adverse cardiac event, a poor or unknown functional capacity, or those in whom stress testing would impact perioperative care).

Given the high cost of preoperative cardiac testing, the potential for delays in care that can adversely impact outcomes, and the recent recommendations, we sought to characterize the rates of inpatient preoperative cardiac testing prior to hip fracture surgery in recent years and to see whether recent recommendations to curb use of these tests were temporally associated with changing rates.

Overview

We utilized two datasets—the Healthcare Cost and Utilization Project (HCUP) State Inpatient Databases (SID) and the American Hospital Association (AHA) Annual Survey—to characterize preoperative cardiac testing. SID data from Maryland, New Jersey, and Washington State from 2011 through September 2015 were used (the ICD coding system changed from ICD9 to ICD10 on October 1). This was combined with AHA data for these years. We included all hospitalizations with a primary ICD9 procedure code for hip fracture repair—78.55, 78.65, 79.05, 79.15, 79.25, 79.35, 79.45, 79.55, 79.65, 79.75, 79.85, and 79.95. We excluded all observations that involved an interhospital transfer. This study was exempt from institutional review board approval.

Measurement and Outcomes

We summarized demographic data for the hospitalizations that met the inclusion criteria as well as the associated hospitals. The primary outcome was the percentage of patients undergoing TTE, stress test, and cardiac catheterization during a hospitalization with a primary procedure code of hip fracture repair. Random effects logistic regression models for each type of diagnostic test were developed to determine the factors that might impact test utilization. In addition to running each test as a separate model, we also performed an analysis in which the outcome was performance of any of these three cardiac tests. Random effects were used to account for clustering of testing within hospitals. Variables included time (3-month intervals), state, age (continuous variable), gender, length of stay, payer (Medicare/Medicaid/private insurance/self-pay/other), hospital teaching status (major teaching/minor teaching/nonteaching), hospital size according to number of beds (continuous variable), and mortality score. Major teaching hospitals are defined as members of the Council of Teaching Hospitals. Minor teaching hospitals are defined as (1) those with one or more postgraduate training programs recognized by the American Council on Graduate Medical Education, (2) those with a medical school affiliation reported to the American Medical Association, or (3) those with an internship or residency approved by the American Osteopathic Association.

The SID has a specific binary indicator variable for each of the three diagnostic tests we evaluated. The use of the diagnostic test is evaluated through both UB-92 revenue codes and ICD9 procedure codes, with the presence of either leading to the indicator variable being positive.¹⁰ Finally, we performed a sensitivity analysis to evaluate the significance of changing utilization trends by interrupted time series analysis. A level of 0.05 was used to determine statistical significance. Analyses were done in STATA 15 (College Station, Texas).

The dataset included 75,144 hospitalizations with a primary procedure code of hip fracture over the study period (Table). The number of hospitalizations per year was fairly consistent over the study period in each state, although there were fewer hospitalizations for 2015 as this included only January through September. The mean age was 72.8 years, and 67% were female. The primary payer was Medicare for 71.7% of hospitalizations. Hospitalizations occurred at 181 hospitals, the plurality of which (42.9%) were minor teaching hospitals. The proportions of hospitalizations that included a TTE, stress test, and cardiac catheterization were 12.6%, 1.1%, and 0.5%, respectively. Overall, 13.5% of patients underwent any cardiac testing.

There was a statistically significantly lower rate of stress tests (odds ratio [OR], 0.32; 95% CI, 0.19-0.54) and cardiac catheterizations (OR, 0.46; 95% CI, 0.27-0.79) in Washington than in Maryland and New Jersey. Female gender was associated with significantly lower adjusted ORs for stress tests (OR, 0.74; 95% CI, 0.63-0.86) and cardiac catheterizations (OR, 0.73; 95% CI, 0.59-0.91), and increasing age was associated with higher adjusted ORs for each test (TTE, OR, 1.033; 95% CI, 1.031-1.035; stress tests, OR, 1.007; 95% CI, 1.001-1.013; cardiac catheterizations, OR, 1.011; 95% CI, 1.003-1.019). Private insurance was associated with a lower likelihood of stress tests (OR, 0.65; 95% CI, 0.50-0.85) and cardiac catheterizations (OR, 0.67; 95% CI,0.46-0.98), and self-pay was associated with a lower likelihood of TTE (OR, 0.76; 95% CI, 0.61-0.95) and stress test (OR, 0.43; 95% CI, 0.21-0.90), all compared with Medicare.

Larger hospitals were associated with a greater likelihood of cardiac catheterizations (OR, 1.18; 95% CI, 1.03-1.36) and a lower likelihood of TTE (OR, 0.89; 95% CI, 0.82-0.96). An unweighted average of these tests between 2011 and October 2015 showed a modest increase in TTEs and a modest decrease in stress tests and cardiac catheterizations (Figure). A multivariable random effects regression for use of TTEs revealed a significantly increasing trend from 2011 to 2014 (OR, 1.04, P < .0001), but the decreasing trend for 2015 was not statistically significant when analyzed according to quarters or months (for which data from only New Jersey and Washington are available).

There has been limited research into how often preoperative cardiac testing occurs in the inpatient setting. Our aim was to study its prevalence prior to hip fracture surgery during a time period when multiple recommendations had been issued to limit its use. We found rates of ischemic testing (stress tests and cardiac catheterizations) to be appropriately, and perhaps surprisingly, low. Our results on ischemic testing rates are consistent with previous studies, which have focused on the outpatient setting where much of the preoperative workup for nonurgent surgeries occurs. The rate of TTEs was higher than in previous studies of the outpatient preoperative setting, although it is unclear what an optimal rate of TTEs is.

A recent study examining outpatient preoperative stress tests within the 30 days before cataract surgeries, knee arthroscopies, or shoulder arthroscopies found a rate of 2.1% for Medicare fee-for-service patients in 2009 with little regional variation.¹¹ Another evaluation using 2009 Medicare claims data found rates of preoperative TTEs and stress tests to be 0.8% and 0.7%, respectively.¹² They included TTEs and stress tests performed within 30 days of a low- or intermediate-risk surgery. A study analyzing the rate of preoperative TTEs between 2009 and 2014 found that rates varied from 2.0% to 3.4% for commercially insured patients aged 50-64 years and Medicare-advantage patients, respectively, in 2009.¹³ These rates decreased by 7.0% and 12.6% from 2009 to 2014. These studies, like ours, suggest that preoperative cardiac testing has not been a major source of wasteful spending. One explanation for the higher rate of TTEs we observed in the inpatient setting might be that primary care physicians in the outpatient setting are more likely to have historical cardiac testing results compared with physicians in a hospital.

We found that the rate of stress testing and cardiac catheterization in Washington was significantly lower than that in Maryland and New Jersey. This is consistent with a number of measures of healthcare utilization – total Medicare reimbursement in the last six months of life, mean number of hospital days in the last six months of life, and healthcare intensity index—for all of which Washington was below the national mean and Maryland and New Jersey were above it.¹⁴

Finally, we found evidence of a lower rate of preoperative stress tests and cardiac catheterizations for women despite controlling for age and mortality score. Of course, we did not control directly for cardiovascular comorbidities; as a result, there could be residual confounding. However, these results are consistent with previous findings of gender bias in both pharmacologic management of coronary artery disease (CAD)¹⁵ and diagnostic testing for suspected CAD.¹⁶

We focused on hospitalizations with a primary procedure code to surgically treat hip fracture. We are unable to tell if the cardiac testing of these patients had occurred before or after the procedure. However, we suspect that the vast majority were completed for preoperative evaluation. It is likely that a small subset were done to diagnose and manage cardiac complications that either accompanied the hip fracture or occurred postoperatively. Another limitation is that we cannot determine if a patient had one of these tests recently in the emergency department or as an outpatient.

We also chose to include only patients who actually had hip fracture surgery. It is possible that the testing rate is higher for all patients admitted for hip fracture and that some of these patients did not have surgery because of abnormal cardiac testing. However, we suspect that this is a very small fraction given the high degree of morbidity and mortality associated with untreated hip fracture.

We found a low rate of preoperative cardiac testing in patients hospitalized for hip fracture surgery both in the years before and after the issuance of recommendations intended to curb its use. Although it is reassuring that the volume of low-value testing is lower than we expected, these findings highlight the importance of targeting utilization improvement efforts toward low-value tests and procedures that are more heavily used, since further curbing the use of infrequently utilized tests and procedures will have only a modest impact on overall healthcare expenditure. Our findings highlight the necessity that professional organizations ensure that they focus on true areas of inappropriate utilization. These are the areas in which improvements will have a major impact on healthcare spending. Further research should aim to quantify unwarranted cardiac testing for other inpatient surgeries that are less urgent, as the urgency of hip fracture repair may be driving the relatively low utilization of inpatient cardiac testing.

Disclosures

The authors have nothing to disclose.

Funding

This project was supported by the Johns Hopkins Hospitalist Scholars Fund and the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core.

Hip fracture is a common reason for unexpected, urgent inpatient surgery in older patients. In 2005, the incidence of hip fracture was 369.0 and 793.5 per 100,000 in men and women respectively.¹ These numbers declined over the preceding decade, potentially as a result of bisphosphonate use. Age- and risk-adjusted 30-day mortality rates for men and women in 2005 were approximately 10% and 5%, respectively.

Evidence suggests that timely surgical repair of hip fractures improves outcomes, although the optimal timing is controversial. Guidelines from the American College of Surgeons Committee on Trauma from 2015 recommend surgical intervention within 48 hours for geriatric hip fracures.² A 2008 systematic review found that operative delay beyond 48 hours was associated with a 41% increase in 30-day all-cause mortality and a 32% increase in one-year all-cause mortality.³ Recent evidence suggests that the rate of complications begins to increase with delays beyond 24 hours.⁴

There has been a focus over the past decade on overuse of preoperative testing for low- and intermediate-risk surgeries.^5-7 Beginning in 2012, the American Board of Internal Medicine initiated the Choosing Wisely^® campaign in which numerous societies issued recommendations on reducing utilization of various diagnostic tests, a number of which have focused on preoperative tests. Two groups—the American Society of Anesthesiologists (ASA) and the American Society of Echocardiography (ASE)— issued specific recommendations on preoperative cardiac testing.⁸ In February 2013, the ASE recommended avoiding preoperative echocardiograms in patients without a history or symptoms of heart disease. In October 2013, the ASA recommended against transthoracic echocardiogram (TTE), transesophageal echocardiogram (TEE), or stress testing for low- or intermediate-risk noncardiac surgery for patients with stable cardiac disease.

Finally, in 2014, the American College of Cardiology (ACC)/American Heart Association (AHA) issued updated perioperative guidelines for patients undergoing noncardiac surgeries.⁹ They recommended preoperative stress testing only in a small subset of cases (patients with an elevated perioperative risk of major adverse cardiac event, a poor or unknown functional capacity, or those in whom stress testing would impact perioperative care).

Given the high cost of preoperative cardiac testing, the potential for delays in care that can adversely impact outcomes, and the recent recommendations, we sought to characterize the rates of inpatient preoperative cardiac testing prior to hip fracture surgery in recent years and to see whether recent recommendations to curb use of these tests were temporally associated with changing rates.

Overview

We utilized two datasets—the Healthcare Cost and Utilization Project (HCUP) State Inpatient Databases (SID) and the American Hospital Association (AHA) Annual Survey—to characterize preoperative cardiac testing. SID data from Maryland, New Jersey, and Washington State from 2011 through September 2015 were used (the ICD coding system changed from ICD9 to ICD10 on October 1). This was combined with AHA data for these years. We included all hospitalizations with a primary ICD9 procedure code for hip fracture repair—78.55, 78.65, 79.05, 79.15, 79.25, 79.35, 79.45, 79.55, 79.65, 79.75, 79.85, and 79.95. We excluded all observations that involved an interhospital transfer. This study was exempt from institutional review board approval.

Measurement and Outcomes

We summarized demographic data for the hospitalizations that met the inclusion criteria as well as the associated hospitals. The primary outcome was the percentage of patients undergoing TTE, stress test, and cardiac catheterization during a hospitalization with a primary procedure code of hip fracture repair. Random effects logistic regression models for each type of diagnostic test were developed to determine the factors that might impact test utilization. In addition to running each test as a separate model, we also performed an analysis in which the outcome was performance of any of these three cardiac tests. Random effects were used to account for clustering of testing within hospitals. Variables included time (3-month intervals), state, age (continuous variable), gender, length of stay, payer (Medicare/Medicaid/private insurance/self-pay/other), hospital teaching status (major teaching/minor teaching/nonteaching), hospital size according to number of beds (continuous variable), and mortality score. Major teaching hospitals are defined as members of the Council of Teaching Hospitals. Minor teaching hospitals are defined as (1) those with one or more postgraduate training programs recognized by the American Council on Graduate Medical Education, (2) those with a medical school affiliation reported to the American Medical Association, or (3) those with an internship or residency approved by the American Osteopathic Association.

The SID has a specific binary indicator variable for each of the three diagnostic tests we evaluated. The use of the diagnostic test is evaluated through both UB-92 revenue codes and ICD9 procedure codes, with the presence of either leading to the indicator variable being positive.¹⁰ Finally, we performed a sensitivity analysis to evaluate the significance of changing utilization trends by interrupted time series analysis. A level of 0.05 was used to determine statistical significance. Analyses were done in STATA 15 (College Station, Texas).

The dataset included 75,144 hospitalizations with a primary procedure code of hip fracture over the study period (Table). The number of hospitalizations per year was fairly consistent over the study period in each state, although there were fewer hospitalizations for 2015 as this included only January through September. The mean age was 72.8 years, and 67% were female. The primary payer was Medicare for 71.7% of hospitalizations. Hospitalizations occurred at 181 hospitals, the plurality of which (42.9%) were minor teaching hospitals. The proportions of hospitalizations that included a TTE, stress test, and cardiac catheterization were 12.6%, 1.1%, and 0.5%, respectively. Overall, 13.5% of patients underwent any cardiac testing.

There was a statistically significantly lower rate of stress tests (odds ratio [OR], 0.32; 95% CI, 0.19-0.54) and cardiac catheterizations (OR, 0.46; 95% CI, 0.27-0.79) in Washington than in Maryland and New Jersey. Female gender was associated with significantly lower adjusted ORs for stress tests (OR, 0.74; 95% CI, 0.63-0.86) and cardiac catheterizations (OR, 0.73; 95% CI, 0.59-0.91), and increasing age was associated with higher adjusted ORs for each test (TTE, OR, 1.033; 95% CI, 1.031-1.035; stress tests, OR, 1.007; 95% CI, 1.001-1.013; cardiac catheterizations, OR, 1.011; 95% CI, 1.003-1.019). Private insurance was associated with a lower likelihood of stress tests (OR, 0.65; 95% CI, 0.50-0.85) and cardiac catheterizations (OR, 0.67; 95% CI,0.46-0.98), and self-pay was associated with a lower likelihood of TTE (OR, 0.76; 95% CI, 0.61-0.95) and stress test (OR, 0.43; 95% CI, 0.21-0.90), all compared with Medicare.

Larger hospitals were associated with a greater likelihood of cardiac catheterizations (OR, 1.18; 95% CI, 1.03-1.36) and a lower likelihood of TTE (OR, 0.89; 95% CI, 0.82-0.96). An unweighted average of these tests between 2011 and October 2015 showed a modest increase in TTEs and a modest decrease in stress tests and cardiac catheterizations (Figure). A multivariable random effects regression for use of TTEs revealed a significantly increasing trend from 2011 to 2014 (OR, 1.04, P < .0001), but the decreasing trend for 2015 was not statistically significant when analyzed according to quarters or months (for which data from only New Jersey and Washington are available).

There has been limited research into how often preoperative cardiac testing occurs in the inpatient setting. Our aim was to study its prevalence prior to hip fracture surgery during a time period when multiple recommendations had been issued to limit its use. We found rates of ischemic testing (stress tests and cardiac catheterizations) to be appropriately, and perhaps surprisingly, low. Our results on ischemic testing rates are consistent with previous studies, which have focused on the outpatient setting where much of the preoperative workup for nonurgent surgeries occurs. The rate of TTEs was higher than in previous studies of the outpatient preoperative setting, although it is unclear what an optimal rate of TTEs is.

A recent study examining outpatient preoperative stress tests within the 30 days before cataract surgeries, knee arthroscopies, or shoulder arthroscopies found a rate of 2.1% for Medicare fee-for-service patients in 2009 with little regional variation.¹¹ Another evaluation using 2009 Medicare claims data found rates of preoperative TTEs and stress tests to be 0.8% and 0.7%, respectively.¹² They included TTEs and stress tests performed within 30 days of a low- or intermediate-risk surgery. A study analyzing the rate of preoperative TTEs between 2009 and 2014 found that rates varied from 2.0% to 3.4% for commercially insured patients aged 50-64 years and Medicare-advantage patients, respectively, in 2009.¹³ These rates decreased by 7.0% and 12.6% from 2009 to 2014. These studies, like ours, suggest that preoperative cardiac testing has not been a major source of wasteful spending. One explanation for the higher rate of TTEs we observed in the inpatient setting might be that primary care physicians in the outpatient setting are more likely to have historical cardiac testing results compared with physicians in a hospital.

We found that the rate of stress testing and cardiac catheterization in Washington was significantly lower than that in Maryland and New Jersey. This is consistent with a number of measures of healthcare utilization – total Medicare reimbursement in the last six months of life, mean number of hospital days in the last six months of life, and healthcare intensity index—for all of which Washington was below the national mean and Maryland and New Jersey were above it.¹⁴

Finally, we found evidence of a lower rate of preoperative stress tests and cardiac catheterizations for women despite controlling for age and mortality score. Of course, we did not control directly for cardiovascular comorbidities; as a result, there could be residual confounding. However, these results are consistent with previous findings of gender bias in both pharmacologic management of coronary artery disease (CAD)¹⁵ and diagnostic testing for suspected CAD.¹⁶

We focused on hospitalizations with a primary procedure code to surgically treat hip fracture. We are unable to tell if the cardiac testing of these patients had occurred before or after the procedure. However, we suspect that the vast majority were completed for preoperative evaluation. It is likely that a small subset were done to diagnose and manage cardiac complications that either accompanied the hip fracture or occurred postoperatively. Another limitation is that we cannot determine if a patient had one of these tests recently in the emergency department or as an outpatient.

We also chose to include only patients who actually had hip fracture surgery. It is possible that the testing rate is higher for all patients admitted for hip fracture and that some of these patients did not have surgery because of abnormal cardiac testing. However, we suspect that this is a very small fraction given the high degree of morbidity and mortality associated with untreated hip fracture.

We found a low rate of preoperative cardiac testing in patients hospitalized for hip fracture surgery both in the years before and after the issuance of recommendations intended to curb its use. Although it is reassuring that the volume of low-value testing is lower than we expected, these findings highlight the importance of targeting utilization improvement efforts toward low-value tests and procedures that are more heavily used, since further curbing the use of infrequently utilized tests and procedures will have only a modest impact on overall healthcare expenditure. Our findings highlight the necessity that professional organizations ensure that they focus on true areas of inappropriate utilization. These are the areas in which improvements will have a major impact on healthcare spending. Further research should aim to quantify unwarranted cardiac testing for other inpatient surgeries that are less urgent, as the urgency of hip fracture repair may be driving the relatively low utilization of inpatient cardiac testing.

Disclosures

The authors have nothing to disclose.

Funding

This project was supported by the Johns Hopkins Hospitalist Scholars Fund and the Johns Hopkins School of Medicine Biostatistics, Epidemiology and Data Management (BEAD) Core.

Advance care planning (ACP) is the process wherein patients, in discussions with their healthcare providers, family members, and other loved ones, make individual decisions about their future healthcare or prepare proxies to guide future medical treatment decisions.^1,2 In 2016, the Centers for Medicare and Medicaid Services (CMS) began paying providers for ACP by using billing codes 99497 (first 30 min of ACP) and 99498 (additional 30 min of ACP). According to the CMS, during the first year after the billing codes were introduced, 22,864 providers billed for ACP conversations with 574,621 patients.³ While all adults are eligible, common triggers for ACP include advanced age, serious illness, and functional status changes that confer an increased risk of dying. We explored the early uptake of the ACP billing code in a large national physician practice that provided mandatory education in use of the ACP billing code, offered a small financial incentive for ACP documentation, and primed physicians to reflect on the patient’s risk of dying in the next year at the time of hospital admission.

We analyzed ACP billing for hospitalized adults aged 65 years or above and who were managed by a large national physician practice that employs acute care providers in hospital medicine, emergency medicine and critical care between January 1, 2017 and March 31, 2017. This practice employs approximately 2,500 hospital-based physicians in 250 community hospitals in 38 states. They collect data through handheld and desktop information technology (IT) tools to facilitate coding, billing, and compliance by hospitalists. Hospitalists receive mandatory web-based training in compliance with CMS ACP billing and templated ACP documentation. Additionally, they receive web-based training in serious illness communication skills during the first two years of employment. The training includes didactic content regarding steps for collaborative decision making, words to use during the encounter, and videos of simulated patient encounters demonstrating best practices. Hospitalists also receive a small financial incentive ($20) for each properly documented ACP conversation that meets CMS criteria for ACP code payment.

Beginning in 2017, hospitalists were required to answer the validated Surprise Question⁴ (SQ; “Would you be surprised if the patient died in the next year?”) for all admitted patients aged 65 years and older. The SQ is useful because it is intuitive and not burdensome for physicians to answer. Moreover, it is predictive of mortality. The pooled prognostic characteristics of the SQ across multiple populations for predicting the outcome of death at 6 months to 18 months include a sensitivity of 67.0% (95% confidence interval [CI] 55.7%-76.7%), a specificity of 80.2% (95% CI 73.3%-85.6%), a positive likelihood ratio of 3.4 (95% CI 2.8–4.1), a negative likelihood ratio of 0.41 (95% CI 0.32-0.54), a positive predictive value of 37.1% (95% CI 30.2%-44.6%), and a negative predictive value of 93.1% (95% CI 91.0%-94.8%).⁵ The SQ primed the admitting physician and triggered an “EoL” (end-of-life) icon next to the patient’s name on the hospitalists’ handheld electronic patient census.

We summarized ACP billing rates and used mixed-effects regression to estimate adjusted ACP rates accounting for patient covariates and clustering at the provider and hospital level. Patient covariates included age; answer to the SQ [“yes,” “no,” or “missing”]); and the presence or absence of seven comorbidities: dementia, heart failure, chronic obstructive pulmonary disease, renal failure, liver failure, metastatic cancer, and nonmetastatic cancer. We quantified the magnitude of provider and hospital variation in ACP rates by using the intraclass correlation coefficient (ICC).

In the first quarter of 2017, hospitalists admitted 113,612 patients aged 65 years and older. Hospitalists were prompted to answer the SQ for 73,731 (65%) of the patients. They were not prompted to answer the SQ for 39,881 (35%) of the patients (ie, missing data for the SQ). Reasons for not prompting include delayed implementation at a site and the patient not being admitted to the hospital (eg, managed on observation status). When prompted, hospitalists answered “no” to the SQ for 41,276/73,731 (56%) of admissions.

Only 6,146/113,612 (5.4%) of all admissions involved a billed ACP conversation. Rates were highest among SQ-prompted/answer “no” cases (8.3%) compared with SQ-prompted/answer “yes” cases (4.1%) and non-SQ-prompted cases (3.5%), with all pairwise differences being statistically significant (P values “yes” vs “no” = .0079, “yes” vs not prompted = .0043, “no” vs not prompted < .0001; see Table 1).

In this large national hospital-based physician practice group, the rates of ACP among acute care patients 65 years of age and older were very low despite the use of education and IT- and incentive-based strategies to encourage ACP conversations among seriously ill older adults. Priming physicians to reflect on the patient’s risk of dying at the time of admission was associated with the doubling of ACP rates.

Despite some lawmakers’ concerns that the ACP billing code may be overused and therefore become a financial burden to the Medicare program⁶, we find the very low use of ACP billing in a population for whom having goals of care conversations is critical—seriously ill older adults who the physician would not be surprised if they died in the next year. This gap is significant because these ACP conversations, when they did occur, were associated with a comfort-focused trajectory, including a more than four-fold increase in hospice referral at discharge.

Causal inference is limited because of the observational nature of the study. While we hypothesize that priming the physicians to reflect on prognosis activated them to prioritize ACP, based on a prior scenario-based randomized trial,⁷ illness severity likely drives ACP conversations. Specifically, patients on observation status (who had missing SQ data) and those for whom the physician answered “yes” to the SQ are less sick than other patients. Additional decision-making heuristics in addition to mortality risk may influence ACP conversations, as suggested by the independent influence of diagnoses, such as dementia or cancer, on ACP. Notably, however, the large amounts of unexplained variation at the physician and the hospital levels exceed the amounts explained by any individual observed patient factor.

Other key limitations of this study include the use of ACP billing as a primary outcome rather than observed and documented ACP conversations and the lack of information on the quality of ACP conversations. These findings reflect the uptake of ACP billing rates soon after the code was introduced. ACP billing rates have likely increased since the first quarter of 2017. Future work should explore diffusion and variation in physician-specific use over time. Finally, despite the nationwide sample, findings may not be generalizable to hospitalists who have not received training and financial incentives for ACP billing.

This study reinforces the possibility that variation in ACP conversations may contribute to variation in end-of-life treatment intensity between providers.^8-10 Low ACP rates among even those with high hospitalist-predicted mortality risk and considerable between-provider variation underscore the need for quality improvement interventions to increase hospital-based ACP.

Acknowledgments

The authors thank Jared Wasserman, Maxwell Bessler, Devon Zoller MD, Mark Rudolph MD, Kristi Franz, and Weiping Zhou for their research assistance.

Disclosures

The authors have nothing to disclose.

Funding

National Institute on Aging award P01 AG019783

Advance care planning (ACP) is the process wherein patients, in discussions with their healthcare providers, family members, and other loved ones, make individual decisions about their future healthcare or prepare proxies to guide future medical treatment decisions.^1,2 In 2016, the Centers for Medicare and Medicaid Services (CMS) began paying providers for ACP by using billing codes 99497 (first 30 min of ACP) and 99498 (additional 30 min of ACP). According to the CMS, during the first year after the billing codes were introduced, 22,864 providers billed for ACP conversations with 574,621 patients.³ While all adults are eligible, common triggers for ACP include advanced age, serious illness, and functional status changes that confer an increased risk of dying. We explored the early uptake of the ACP billing code in a large national physician practice that provided mandatory education in use of the ACP billing code, offered a small financial incentive for ACP documentation, and primed physicians to reflect on the patient’s risk of dying in the next year at the time of hospital admission.

We analyzed ACP billing for hospitalized adults aged 65 years or above and who were managed by a large national physician practice that employs acute care providers in hospital medicine, emergency medicine and critical care between January 1, 2017 and March 31, 2017. This practice employs approximately 2,500 hospital-based physicians in 250 community hospitals in 38 states. They collect data through handheld and desktop information technology (IT) tools to facilitate coding, billing, and compliance by hospitalists. Hospitalists receive mandatory web-based training in compliance with CMS ACP billing and templated ACP documentation. Additionally, they receive web-based training in serious illness communication skills during the first two years of employment. The training includes didactic content regarding steps for collaborative decision making, words to use during the encounter, and videos of simulated patient encounters demonstrating best practices. Hospitalists also receive a small financial incentive ($20) for each properly documented ACP conversation that meets CMS criteria for ACP code payment.

Beginning in 2017, hospitalists were required to answer the validated Surprise Question⁴ (SQ; “Would you be surprised if the patient died in the next year?”) for all admitted patients aged 65 years and older. The SQ is useful because it is intuitive and not burdensome for physicians to answer. Moreover, it is predictive of mortality. The pooled prognostic characteristics of the SQ across multiple populations for predicting the outcome of death at 6 months to 18 months include a sensitivity of 67.0% (95% confidence interval [CI] 55.7%-76.7%), a specificity of 80.2% (95% CI 73.3%-85.6%), a positive likelihood ratio of 3.4 (95% CI 2.8–4.1), a negative likelihood ratio of 0.41 (95% CI 0.32-0.54), a positive predictive value of 37.1% (95% CI 30.2%-44.6%), and a negative predictive value of 93.1% (95% CI 91.0%-94.8%).⁵ The SQ primed the admitting physician and triggered an “EoL” (end-of-life) icon next to the patient’s name on the hospitalists’ handheld electronic patient census.

We summarized ACP billing rates and used mixed-effects regression to estimate adjusted ACP rates accounting for patient covariates and clustering at the provider and hospital level. Patient covariates included age; answer to the SQ [“yes,” “no,” or “missing”]); and the presence or absence of seven comorbidities: dementia, heart failure, chronic obstructive pulmonary disease, renal failure, liver failure, metastatic cancer, and nonmetastatic cancer. We quantified the magnitude of provider and hospital variation in ACP rates by using the intraclass correlation coefficient (ICC).

In the first quarter of 2017, hospitalists admitted 113,612 patients aged 65 years and older. Hospitalists were prompted to answer the SQ for 73,731 (65%) of the patients. They were not prompted to answer the SQ for 39,881 (35%) of the patients (ie, missing data for the SQ). Reasons for not prompting include delayed implementation at a site and the patient not being admitted to the hospital (eg, managed on observation status). When prompted, hospitalists answered “no” to the SQ for 41,276/73,731 (56%) of admissions.

Only 6,146/113,612 (5.4%) of all admissions involved a billed ACP conversation. Rates were highest among SQ-prompted/answer “no” cases (8.3%) compared with SQ-prompted/answer “yes” cases (4.1%) and non-SQ-prompted cases (3.5%), with all pairwise differences being statistically significant (P values “yes” vs “no” = .0079, “yes” vs not prompted = .0043, “no” vs not prompted < .0001; see Table 1).

In this large national hospital-based physician practice group, the rates of ACP among acute care patients 65 years of age and older were very low despite the use of education and IT- and incentive-based strategies to encourage ACP conversations among seriously ill older adults. Priming physicians to reflect on the patient’s risk of dying at the time of admission was associated with the doubling of ACP rates.

Despite some lawmakers’ concerns that the ACP billing code may be overused and therefore become a financial burden to the Medicare program⁶, we find the very low use of ACP billing in a population for whom having goals of care conversations is critical—seriously ill older adults who the physician would not be surprised if they died in the next year. This gap is significant because these ACP conversations, when they did occur, were associated with a comfort-focused trajectory, including a more than four-fold increase in hospice referral at discharge.

Causal inference is limited because of the observational nature of the study. While we hypothesize that priming the physicians to reflect on prognosis activated them to prioritize ACP, based on a prior scenario-based randomized trial,⁷ illness severity likely drives ACP conversations. Specifically, patients on observation status (who had missing SQ data) and those for whom the physician answered “yes” to the SQ are less sick than other patients. Additional decision-making heuristics in addition to mortality risk may influence ACP conversations, as suggested by the independent influence of diagnoses, such as dementia or cancer, on ACP. Notably, however, the large amounts of unexplained variation at the physician and the hospital levels exceed the amounts explained by any individual observed patient factor.

Other key limitations of this study include the use of ACP billing as a primary outcome rather than observed and documented ACP conversations and the lack of information on the quality of ACP conversations. These findings reflect the uptake of ACP billing rates soon after the code was introduced. ACP billing rates have likely increased since the first quarter of 2017. Future work should explore diffusion and variation in physician-specific use over time. Finally, despite the nationwide sample, findings may not be generalizable to hospitalists who have not received training and financial incentives for ACP billing.

This study reinforces the possibility that variation in ACP conversations may contribute to variation in end-of-life treatment intensity between providers.^8-10 Low ACP rates among even those with high hospitalist-predicted mortality risk and considerable between-provider variation underscore the need for quality improvement interventions to increase hospital-based ACP.

Acknowledgments

The authors thank Jared Wasserman, Maxwell Bessler, Devon Zoller MD, Mark Rudolph MD, Kristi Franz, and Weiping Zhou for their research assistance.

Disclosures

The authors have nothing to disclose.

Funding

National Institute on Aging award P01 AG019783

Advance care planning (ACP) is the process wherein patients, in discussions with their healthcare providers, family members, and other loved ones, make individual decisions about their future healthcare or prepare proxies to guide future medical treatment decisions.^1,2 In 2016, the Centers for Medicare and Medicaid Services (CMS) began paying providers for ACP by using billing codes 99497 (first 30 min of ACP) and 99498 (additional 30 min of ACP). According to the CMS, during the first year after the billing codes were introduced, 22,864 providers billed for ACP conversations with 574,621 patients.³ While all adults are eligible, common triggers for ACP include advanced age, serious illness, and functional status changes that confer an increased risk of dying. We explored the early uptake of the ACP billing code in a large national physician practice that provided mandatory education in use of the ACP billing code, offered a small financial incentive for ACP documentation, and primed physicians to reflect on the patient’s risk of dying in the next year at the time of hospital admission.

We analyzed ACP billing for hospitalized adults aged 65 years or above and who were managed by a large national physician practice that employs acute care providers in hospital medicine, emergency medicine and critical care between January 1, 2017 and March 31, 2017. This practice employs approximately 2,500 hospital-based physicians in 250 community hospitals in 38 states. They collect data through handheld and desktop information technology (IT) tools to facilitate coding, billing, and compliance by hospitalists. Hospitalists receive mandatory web-based training in compliance with CMS ACP billing and templated ACP documentation. Additionally, they receive web-based training in serious illness communication skills during the first two years of employment. The training includes didactic content regarding steps for collaborative decision making, words to use during the encounter, and videos of simulated patient encounters demonstrating best practices. Hospitalists also receive a small financial incentive ($20) for each properly documented ACP conversation that meets CMS criteria for ACP code payment.

Beginning in 2017, hospitalists were required to answer the validated Surprise Question⁴ (SQ; “Would you be surprised if the patient died in the next year?”) for all admitted patients aged 65 years and older. The SQ is useful because it is intuitive and not burdensome for physicians to answer. Moreover, it is predictive of mortality. The pooled prognostic characteristics of the SQ across multiple populations for predicting the outcome of death at 6 months to 18 months include a sensitivity of 67.0% (95% confidence interval [CI] 55.7%-76.7%), a specificity of 80.2% (95% CI 73.3%-85.6%), a positive likelihood ratio of 3.4 (95% CI 2.8–4.1), a negative likelihood ratio of 0.41 (95% CI 0.32-0.54), a positive predictive value of 37.1% (95% CI 30.2%-44.6%), and a negative predictive value of 93.1% (95% CI 91.0%-94.8%).⁵ The SQ primed the admitting physician and triggered an “EoL” (end-of-life) icon next to the patient’s name on the hospitalists’ handheld electronic patient census.

We summarized ACP billing rates and used mixed-effects regression to estimate adjusted ACP rates accounting for patient covariates and clustering at the provider and hospital level. Patient covariates included age; answer to the SQ [“yes,” “no,” or “missing”]); and the presence or absence of seven comorbidities: dementia, heart failure, chronic obstructive pulmonary disease, renal failure, liver failure, metastatic cancer, and nonmetastatic cancer. We quantified the magnitude of provider and hospital variation in ACP rates by using the intraclass correlation coefficient (ICC).

In the first quarter of 2017, hospitalists admitted 113,612 patients aged 65 years and older. Hospitalists were prompted to answer the SQ for 73,731 (65%) of the patients. They were not prompted to answer the SQ for 39,881 (35%) of the patients (ie, missing data for the SQ). Reasons for not prompting include delayed implementation at a site and the patient not being admitted to the hospital (eg, managed on observation status). When prompted, hospitalists answered “no” to the SQ for 41,276/73,731 (56%) of admissions.

Only 6,146/113,612 (5.4%) of all admissions involved a billed ACP conversation. Rates were highest among SQ-prompted/answer “no” cases (8.3%) compared with SQ-prompted/answer “yes” cases (4.1%) and non-SQ-prompted cases (3.5%), with all pairwise differences being statistically significant (P values “yes” vs “no” = .0079, “yes” vs not prompted = .0043, “no” vs not prompted < .0001; see Table 1).

In this large national hospital-based physician practice group, the rates of ACP among acute care patients 65 years of age and older were very low despite the use of education and IT- and incentive-based strategies to encourage ACP conversations among seriously ill older adults. Priming physicians to reflect on the patient’s risk of dying at the time of admission was associated with the doubling of ACP rates.

Despite some lawmakers’ concerns that the ACP billing code may be overused and therefore become a financial burden to the Medicare program⁶, we find the very low use of ACP billing in a population for whom having goals of care conversations is critical—seriously ill older adults who the physician would not be surprised if they died in the next year. This gap is significant because these ACP conversations, when they did occur, were associated with a comfort-focused trajectory, including a more than four-fold increase in hospice referral at discharge.

Causal inference is limited because of the observational nature of the study. While we hypothesize that priming the physicians to reflect on prognosis activated them to prioritize ACP, based on a prior scenario-based randomized trial,⁷ illness severity likely drives ACP conversations. Specifically, patients on observation status (who had missing SQ data) and those for whom the physician answered “yes” to the SQ are less sick than other patients. Additional decision-making heuristics in addition to mortality risk may influence ACP conversations, as suggested by the independent influence of diagnoses, such as dementia or cancer, on ACP. Notably, however, the large amounts of unexplained variation at the physician and the hospital levels exceed the amounts explained by any individual observed patient factor.

Other key limitations of this study include the use of ACP billing as a primary outcome rather than observed and documented ACP conversations and the lack of information on the quality of ACP conversations. These findings reflect the uptake of ACP billing rates soon after the code was introduced. ACP billing rates have likely increased since the first quarter of 2017. Future work should explore diffusion and variation in physician-specific use over time. Finally, despite the nationwide sample, findings may not be generalizable to hospitalists who have not received training and financial incentives for ACP billing.

This study reinforces the possibility that variation in ACP conversations may contribute to variation in end-of-life treatment intensity between providers.^8-10 Low ACP rates among even those with high hospitalist-predicted mortality risk and considerable between-provider variation underscore the need for quality improvement interventions to increase hospital-based ACP.

Acknowledgments

The authors thank Jared Wasserman, Maxwell Bessler, Devon Zoller MD, Mark Rudolph MD, Kristi Franz, and Weiping Zhou for their research assistance.

Disclosures

The authors have nothing to disclose.

Funding

National Institute on Aging award P01 AG019783