Kathryn Goggins, MPH

Despite increased awareness of Choosing Wisely (CW)^® recommendations to reduce low-value care,¹ there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending.^2-6 Implementation science seeks to accelerate the translation of evidence-based interventions into clinical practice and the deimplementation of low-value care.^7-9 Based on established principles of implementation science, we used a prospective, nonrandomized study design to assess a CW intervention to reduce chest X-ray (CXR) ordering in adult intensive care units (ICUs).¹⁰

In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.¹¹ Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.^12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.^11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.^18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.

METHODS

Setting and Design

Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.

We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.

Intervention

Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”¹⁹—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). ²⁰

 

In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.

In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.

To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).²⁰ Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.

Quantitative Evaluation

We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.

The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.²¹

Qualitative Evaluation

Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.^22,23

Embedded Observation

From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).^22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.

 

Semistructured Interviews

Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).

Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.

RESULTS

Quantitative Results

We analyzed CXR ordering data from a period of 86 weeks, comprising 50 weeks of baseline data, three weeks of washout period, and 33 weeks following the introduction of the intervention. In both ICUs, patient characteristics were similar in the baseline and intervention periods (Table 1).

Cardiovascular Intensive Care Unit

The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.

Medical Intensive Care Unit

For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.

Qualitative Results

We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.

Major differences between the units pertained to the “inner setting” domain. Compared with the CVICU, which is staffed by a stable group of ACNPs, two of the three MICU teams are staffed by resident physicians who rotate on and off service. CVICU providers and ACNPs in the MICU reported significant investment in the CXR and other QI interventions. Conversely, resident physicians, who complete two- to four-week MICU rotations, reported less investment as well as greater fatigue and competing priorities. Some MICU residents began ignoring weekly feedback, citing “e-mail fatigue” and the lack of in-person follow-up or didactic sessions associated with the reports.

We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.

Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.²⁴ We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).

Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.

 

DISCUSSION

A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.

Relatively few studies have demonstrated effective interventions that address CW recommendations.^25-28 However, three population-level analyses of insurance claims show mixed results.^3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.^2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.

Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.

As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.³⁰ Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.^30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.

To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.³³ We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).^34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.

This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.³⁶ Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.

Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.³⁷ Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.

In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.³⁸ Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.

 

Acknowledgments

The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.

Disclosures

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.

Funding

This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.

 

Despite increased awareness of Choosing Wisely (CW)^® recommendations to reduce low-value care,¹ there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending.^2-6 Implementation science seeks to accelerate the translation of evidence-based interventions into clinical practice and the deimplementation of low-value care.^7-9 Based on established principles of implementation science, we used a prospective, nonrandomized study design to assess a CW intervention to reduce chest X-ray (CXR) ordering in adult intensive care units (ICUs).¹⁰

In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.¹¹ Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.^12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.^11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.^18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.

METHODS

Setting and Design

Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.

We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.

Intervention

Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”¹⁹—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). ²⁰

 

In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.

In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.

To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).²⁰ Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.

Quantitative Evaluation

We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.

The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.²¹

Qualitative Evaluation

Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.^22,23

Embedded Observation

From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).^22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.

 

Semistructured Interviews

Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).

Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.

RESULTS

Quantitative Results

We analyzed CXR ordering data from a period of 86 weeks, comprising 50 weeks of baseline data, three weeks of washout period, and 33 weeks following the introduction of the intervention. In both ICUs, patient characteristics were similar in the baseline and intervention periods (Table 1).

Cardiovascular Intensive Care Unit

The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.

Medical Intensive Care Unit

For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.

Qualitative Results

We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.

Major differences between the units pertained to the “inner setting” domain. Compared with the CVICU, which is staffed by a stable group of ACNPs, two of the three MICU teams are staffed by resident physicians who rotate on and off service. CVICU providers and ACNPs in the MICU reported significant investment in the CXR and other QI interventions. Conversely, resident physicians, who complete two- to four-week MICU rotations, reported less investment as well as greater fatigue and competing priorities. Some MICU residents began ignoring weekly feedback, citing “e-mail fatigue” and the lack of in-person follow-up or didactic sessions associated with the reports.

We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.

Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.²⁴ We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).

Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.

 

DISCUSSION

A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.

Relatively few studies have demonstrated effective interventions that address CW recommendations.^25-28 However, three population-level analyses of insurance claims show mixed results.^3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.^2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.

Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.

As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.³⁰ Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.^30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.

To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.³³ We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).^34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.

This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.³⁶ Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.

Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.³⁷ Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.

In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.³⁸ Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.

 

Acknowledgments

The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.

Disclosures

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.

Funding

This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.

 

Despite increased awareness of Choosing Wisely (CW)^® recommendations to reduce low-value care,¹ there is limited published data about strategies to implement these guidelines or evidence that they have influenced ordering patterns or reduced healthcare spending.^2-6 Implementation science seeks to accelerate the translation of evidence-based interventions into clinical practice and the deimplementation of low-value care.^7-9 Based on established principles of implementation science, we used a prospective, nonrandomized study design to assess a CW intervention to reduce chest X-ray (CXR) ordering in adult intensive care units (ICUs).¹⁰

In ICUs, CXR ordering strategies may be routine (daily) or on-demand (with clinical indication). The former strategy’s principal advantage is the potential to detect life-threatening situations that may otherwise escape diagnosis.¹¹ Disadvantages include cost, radiation exposure, patient inconvenience, false-positive workups, and low diagnostic and therapeutic value.^12,13 On-demand strategies may safely reduce CXR ordering by 32% to 45%.^11-17 Based on this evidence, the Critical Care Societies Collaborative and the American College of Radiology have recommended on-demand CXR ordering.^18,19 Here, we describe the effectiveness of an intervention to reduce CXR ordering in two ICUs while evaluating the deimplementation strategies using a validated framework.

METHODS

Setting and Design

Vanderbilt University Medical Center (VUMC) is an academic referral center in Nashville, Tennessee. The cardiovascular ICU (CVICU) has 27 beds and the medical ICU (MICU) has 34 beds. Acute care nurse practitioners (ACNPs) and two critical care physicians staff the CVICU; cardiology fellows, anesthesia critical care fellows, and transplant and cardiac surgeons are also active in patient care. The MICU is staffed by two critical care physicians who supervise one team of ACNPs and two teams of medical residents who rotate through the unit every two weeks. Each MICU team is assigned a fellow in pulmonary and critical care.

We conducted a prospective, nonrandomized study in these units from October 2015 to June 2016. The VUMC Institutional Review Board approved the intervention as a quality improvement (QI) activity, waiving the requirement for informed consent.

Intervention

Following the top CW recommendation of the Critical Care Societies Collaborative—“Don’t order diagnostic tests at regular intervals (such as every day), but rather in response to specific clinical questions.”¹⁹—the VUMC resident-led CW Steering Committee designed a multifaceted approach to reduce ordering of routine CXRs in ICUs. The intervention included a didactic session on CW and proper CXR ordering practices, peer champions, data audits, and feedback to providers through weekly e-mails (see Supplemental Materials, 1 – Resident Presentation and 2 – CXR Flyer). ²⁰

 

In September 2015, CVICU and MICU teams received a didactic session highlighting CW, current CXR ordering rates, and the plan for reducing CXR ordering. On October 5, 2015, teams began receiving weekly e-mails with ordering rates defined as CXRs ordered per patient per day and a brief rationale for reducing unnecessary CXRs. To encourage friendly competition, we provided weekly rates to the MICU teams, allowing for transparent benchmarking against one another. A similar competition strategy was not used in the CVICU due to the lack of multiple teams.

In the CVICU, two ACNPs volunteered as peer champions. These champions coordinated data feedback and advocated for the intervention among their colleagues. In the MICU, three internal medicine residents volunteered as peer champions and fulfilled similar roles.

To facilitate deimplementation, we conducted two Plan-Do-Study-Act (PDSA) cycles, the first from November to mid-December 2015 and the second from mid-December 2015 to mid-January 2016. During these cycles, we tailored our deimplementation strategy based on barriers identified by the peer champions and ICU leaders (described in the Qualitative Results section). Peer champions and the CW Steering Committee generated potential solutions by conversing with stakeholders and using the Expert Recommendations for Implementing Change (ERIC).²⁰ Interventions included disseminating promotional flyers, holding meetings with stakeholders, and providing monthly CXR ordering rates. After the PDSA cycles, we continued reexamining the deimplementation efforts by reviewing ordering rates and soliciting feedback from ICU leaders and peer champions. However, no significant changes to the intervention were made during this time.

Quantitative Evaluation

We extracted data from VUMC’s Enterprise Data Warehouse during the intervention period (October 5, 2015 to May 24, 2016) and a historical control period (October 1, 2014 to October 4, 2015). Within each ICU, descriptive statistics were used to compare patient cohorts in the baseline and intervention periods by age, sex, and race.

The primary outcome was CXRs ordered per patient per day by hospital unit (CVICU or MICU). The baseline period included all data between October 1, 2014 and September 15, 2015. To account for priming of providers from didactic education, we allowed a washout period from September 16, 2015 to October 4, 2015. As a preliminary analysis, we compared CXR rates in the baseline and intervention periods using Wilcoxon rank-sum tests. We then conducted interrupted time-series analyses with segmented linear regression to assess differences in linear trends in CXR rates over the two periods. To account for different staffing models in the MICU, we stratified the impact of the intervention by team—medical resident (physician) or ACNP. R version 3.4.0 was used for statistical analysis.²¹

Qualitative Evaluation

Our qualitative evaluation consisted of embedded observation and semistructured interviews with stakeholders. The qualitative portion was guided by the Consolidated Framework for Implementation Research (CFIR), a widely used framework for design and evaluation of improvement initiatives that helped us to determine major facilitators and barriers to implementation.^22,23

Embedded Observation

From November 2015 to January 2016, we observed morning rounds in the CVICU and MICU one to two times weekly to understand factors facilitating and inhibiting uptake of the intervention. Observations were recorded and organized using a CFIR-based template and directed toward understanding interactions among team members (eg, the decision-making process hierarchy), team workflows and decision-making processes, process of ordering CXRs, and providers’ knowledge and perceptions of the CXR intervention (see Supplemental Material, 3 – CFIR Table).^22,23 After rounds, ICU team members were invited to share suggestions for improving the intervention. All observations occurred during and shortly following morning rounds when the vast majority of routine CXRs are ordered; we did not evaluate night or evening workflows. In the spirit of continuous improvement, we evaluated data in real-time.

 

Semistructured Interviews

Based on the direct observations, we developed semistructured interview questions to further evaluate provider perspectives (eg, “Do you believe ICU patients need a daily CXR?”) and constructs aligning with CFIR (eg, “intervention source—internally vs externally developed;” see Supplemental Material, 4 – Interview Questions).

Stakeholders from both ICUs were recruited through e-mail and in-person requests to participate in semistructured interviews. In the CVICU, we interviewed critical care physicians, anesthesia critical care fellows, and ACNPs. In the MICU, we interviewed medical students, interns, residents, critical care fellows, attending intensivist physicians, and ACNPs. We also interviewed X-ray technologists who routinely perform portable films in the units.

RESULTS

Quantitative Results

We analyzed CXR ordering data from a period of 86 weeks, comprising 50 weeks of baseline data, three weeks of washout period, and 33 weeks following the introduction of the intervention. In both ICUs, patient characteristics were similar in the baseline and intervention periods (Table 1).

Cardiovascular Intensive Care Unit

The median baseline CXR ordering rate in the CVICU was 1.16 CXRs per patient per day, with interquartile range (IQR) 1.06-1.28. During the intervention period, the rate dropped to 1.07 (IQR 0.94-1.21; P < .001; Table 2). The time-series analysis suggested an essentially flat trend during the baseline period, followed by a small but significant drop in ordering rates during the intervention period (P < .001; Table 3 and Figure 1). Ordering rates appeared to increase slightly over the course of the intervention period, but this slight upward trend was not significantly different from the flat trend seen during the baseline period.

Medical Intensive Care Unit

For both physician and ACPN teams, the median baseline CXR ordering rates in the MICU were much lower than the baseline rate in the CVICU (Table 2). For the MICU physician care team, the baseline CXR ordering rate was 0.60 CXRs per patient per day (IQR 0.48-0.73). For the ACNP team, the median rate was 0.39 CXRs per patient per day (IQR 0.21-0.57). Both rates stayed approximately the same during the intervention period (Table 2). The time-series analysis suggested a statistically significant but very slight downward trend in CXR ordering rates during the baseline period, in the physician (P = .011) and ACNP (P = .022) teams (Table 3, Figure 2). Under this model, a small increase in CXR ordering initially occurred during the intervention period for both physician and ACNP teams (P = .010 and P = .055, respectively), after which the rates declined slightly. Trends in ordering rates during the intervention period were not significantly different from the slight downward trends seen during the baseline period.

Qualitative Results

We identified 25 of 39 CFIR constructs as relevant to the initiative (see Supplemental Materials, 3 – CFIR Table.) We determined the major facilitators of deimplementation to be peer champion discussions about CXR ordering on rounds and weekly data feedback, particularly if accompanied by in-person follow-up.

Major differences between the units pertained to the “inner setting” domain. Compared with the CVICU, which is staffed by a stable group of ACNPs, two of the three MICU teams are staffed by resident physicians who rotate on and off service. CVICU providers and ACNPs in the MICU reported significant investment in the CXR and other QI interventions. Conversely, resident physicians, who complete two- to four-week MICU rotations, reported less investment as well as greater fatigue and competing priorities. Some MICU residents began ignoring weekly feedback, citing “e-mail fatigue” and the lack of in-person follow-up or didactic sessions associated with the reports.

We also noted differences in CXR ordering rationales and decisions between the units. Generally, residents in the MICU and ACNPs in the CVICU made decisions to order CXRs. However, decisions were influenced by the expectations of attending physicians. While CVICU providers tended to order CXRs reflexively as part of morning labs, MICU providers—in particular, ACNPs who had been trained on indications for proper CXR ordering—generally ordered CXRs for specific indications (eg, worsening respiratory status). Of note, MICU ACNPs reported the use of bedside ultrasound as an alternate imaging modality and a reason for their higher threshold to order CXRs.

Deimplementation barriers in both units included the need to identify goal CXR ordering rates and the intervention’s limited visibility. To address these barriers, we conducted PDSA cycles and used the CFIR and ERIC to generate potential solutions.²⁴ We established a goal of a 20% absolute reduction in the CVICU, added monthly CXR rates to weekly e-mail reports to better account for variations in patient populations and ordering practices, and circulated materials to promote on-demand CXR ordering. Promotional materials contained guidelines on CXR ordering and five “Frequently Held Misconceptions” about ordering practices with succinct, evidence-based explanations (see Supplemental Material, 2 – CXR Flyer).

Approximately four months after the start of intervention, some CVICU physicians became concerned that on-demand CXR ordering might be inappropriate for high-risk surgical patients, including those who are undergoing or have undergone heart transplants, lung transplants, or left-ventricular assist device placement. This concern arose following two adverse outcomes, which were not attributed to the CXR initiative, but which heightened concerns about patient safety. A rise in CXR ordering then occurred, and CVICU providers requested that we perform an analysis of these high-risk groups. While segmented linear regression in this subgroup suggested that average daily CXR ordering rates did decrease among the high-risk group at the start of the intervention period (P = .001), the difference between the rates in the two periods was not significant using the Wilcoxon rank-sum test. Exclusion of these patients from the main analysis did not alter the interpretation of the findings reported above for the CVICU.

 

DISCUSSION

A deimplementation intervention using provider education, peer champions, and data feedback was associated with fewer CXRs in the CVICU (P < .001) but not in the MICU. The CFIR-guided qualitative analysis was valuable for evaluating our deimplementation strategy and for identifying differences between the two ICUs.

Relatively few studies have demonstrated effective interventions that address CW recommendations.^25-28 However, three population-level analyses of insurance claims show mixed results.^3,4,29 Experts have thus proposed using implementation science to improve uptake of CW recommendations.^2,3,7,8 Our study demonstrates the effectiveness of this approach. As expected, providers largely endorsed an on-demand CXR ordering strategy. Using the CFIR, however, we discovered barriers (eg, concern that data feedback did not reflect variations in patients’ needs). Using methods from implementation science allowed us to diagnose and tailor our approaches.

Our qualitative evaluation suggested that the intervention was ineffective mostly due to CFIR’s “inner setting” constructs, including resident fatigue, competing priorities, and decreased investment in QI projects because of the rotating nature of providers in training. Baseline CXR ordering rates in the MICU were also considerably lower than in the CVICU. We observed that CVICU providers ordered many CXRs following the placement of lines or tubes and that ACNPs in the MICU had received education on appropriate CXR ordering practices and had access to an alternate imaging modality in ultrasound. These factors may partially explain the difference in baseline rates.

As noted in a study of cardiac stress testing guidelines, the existence of high-value care recommendations does not mean overuse.³⁰ Indeed, the lack of significant CXR over-ordering in the MICU highlights the importance of baseline measurement and partnering with information technology departments to create the best possible data feedback systems.^30-32 Our experience shows that these systems should provide sufficient pre-implementation data (ideally >1 year), such that teams selecting QI projects can ensure that a project is a good use of institutional resources and change capital.

To inform future work, we informally assessed program costs and savings. We estimate the initiative cost $1,600, including $1,000 for curriculum development and teaching time, $300 for educational materials, and $300 for CXR tracking dashboard development. Hospital charges and reimbursements for CXR vary widely.³³ We calculated savings using a range of rates, from a conservative $23 (the Medicare reimbursement rate for single-view CXR, CPT code 71010, global fee) to $50 (an approximate blended reimbursement rate across payers).^34,35 In the CVICU, we estimate that 51 CXRs were avoided each month, saving $1,173-$2,550 per month or $9,384-$20,400 over eight months of follow-up. Annualizing these figures, we estimate net savings of $12,476-$29,000 in the first year in a 27-bed ICU. Costs to continue the program include education of new employees, booster training, and dashboard maintenance for an estimated annual cost of $1,000. It is difficult to estimate effectiveness over time, but if we conservatively assume that 30 CXRs were avoided each month, then the projected savings would be $8,280-$18,000 per year or an annual net savings of $7,280-$17,000 in the ICU. Although these amounts are modest, providing trainees with experiential learning opportunities in high-value care is valuable in its own right, meets curricular goals, may result in spill-over effects to other diagnostic and therapeutic decisions, and may influence long-term practice patterns. Institutional decisions to pursue projects such as this should take into account these potential benefits.

This evaluation is not without limitations. First, the study was conducted in a single tertiary-care hospital, potentially limiting its generalizability.³⁶ Second, the study design lacked a concurrent control group, and observed outcomes may have been influenced by broader CXR utilization trends, increased awareness of low-value care generally or from previous CW projects at VUMC, seasonal effects, or the Hawthorne effect. Third, the study outcome was all CXRs ordered, rather than CXRs that were unnecessary or not clinically indicated. We chose all CXRs because it was more pragmatic, did not require clinical case review, and could be incorporated promptly into dashboards, enabling timely performance feedback. Other performance measures have taken a similar tack (eg, tracking all-cause readmissions rather than preventable readmissions). Given this approach, we did not track clinical indications for CXRs (eg, central line placement). Fourth, although we compared resident and APRN orders, we did not collect data on other provider characteristics such as years in/out of training or board certification status. These considerations should be addressed in future research.

Finally, the increase in CVICU CXR ordering at the end of the intervention period, which occurred following two adverse events, raises concerns about sustainability. While unrelated to CXR orders, the events resulted in increased ordering of diagnostic tests and showed the difficulty of deimplementation in ICUs. Indeed, some CVICU providers argued that on-demand CXR ordering represented minimal potential cost savings and had not been studied among heart and lung transplant patients. Subsequently, Tonna et al. have shown that on-demand CXR ordering can be safely implemented among such patients.³⁷ Also similar to our study, Tonna et al. observed an initial decrease in CXR ordering, followed by a gradual increase toward baseline ordering rates. These findings highlight the need for sustained awareness and interventions and for the careful selection of high-value projects.

In conclusion, our study shows that a deimplementation intervention based on CW recommendations can reduce CXR ordering and that ongoing evaluation of contextual factors provides insights for both real-time modifications of current interventions and the design of future interventions. We found that messaging about reducing unnecessary tests works well when discussions are framed at the unit level but may be counterproductive if used to question individual ordering decisions.³⁸ Additional lessons learned include the value of participation on rounds to build trust among stakeholders, the utility of monthly rather than weekly statistics for feedback, stakeholder input and peer champions, and differences in approach with physician and ACNP audiences.

 

Acknowledgments

The authors thank the VUMC Choosing Wisely committee; Mr. Bill Harrell in Advanced Data Analytics for developing the Tableau platform used in our data feedback strategy; Emily Feld, MD, Jerry Zifodya, MD, and Ryan Kindle, MD for assisting with data feedback to providers in the MICU; Todd Rice, MD, Director of the MICU, for his support of the initiative; and Beth Prusaczyk, PhD, MSW and David Stevenson, PhD for providing feedback on earlier drafts of this manuscript.

Disclosures

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, outside the submitted work. All other authors have nothing to disclose.

Funding

This work was supported by an Innovation Grant from the Alliance for Academic Internal Medicine (AAIM, 2016) and by the Departments of Internal Medicine and Graduate Medical Education at Vanderbilt University Medical Center. The AAIM did not have a role in the study design, data collection, data analysis, data interpretation, or manuscript writing.

 

Most studies to identify risk factors for readmission among patients with heart failure (HF) have focused on demographic and clinical characteristics.^1,2 Although easy to extract from administrative databases, this approach fails to capture the complex psychosocial and cognitive factors that influence the ability of HF patients to manage their disease in the postdischarge period, as depicted in the framework by Meyers et al.³ (2014). To date, studies have found low health literacy, decreased social support, and cognitive impairment to be associated with health behaviors and outcomes among HF patients, including decreased self-care,⁴ low HF-specific knowledge,⁵ medication nonadherence,⁶ hospitalizations,⁷ and mortality.^8-10 Less, however, is known about the effect of numeracy on HF outcomes, such as 30-day readmission.

Numeracy, or quantitative literacy, refers to the ability to access, understand, and apply numerical data to health-related decisions.¹¹ It is estimated that 110 million people in the United States have limited numeracy skills.¹² Low numeracy is a risk factor for poor glycemic control among patients with diabetes,¹³ medication adherence in HIV/AIDS,¹⁴ and worse blood pressure control in hypertensives.¹⁵ Much like these conditions, HF requires that patients understand, use, and act on numerical information. Maintaining a low-salt diet, monitoring weight, adjusting diuretic doses, and measuring blood pressure are tasks that HF patients are asked to perform on a daily or near-daily basis. These tasks are particularly important in the posthospitalization period and could be complicated by medication changes, which might create additional challenges for patients with inadequate numeracy. Additionally, cognitive impairment, which is a highly prevalent comorbid condition among adults with HF,^16,17 might impose additional barriers for those with inadequate numeracy who do not have adequate social support. However, to date, numeracy in the context of HF has not been well described.

Herein, we examined the effects of numeracy, alongside health literacy and cognition, on 30-day readmission risk among patients hospitalized for acute decompensated HF (ADHF).

METHODS

Study Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective observational study of patients admitted with cardiovascular disease to Vanderbilt University Medical Center (VUMC), an academic tertiary care hospital. VICS was designed to investigate the impact of social determinants of health on postdischarge health outcomes. A detailed description of the study rationale, design, and methods is described elsewhere.³

Briefly, participants completed a baseline interview while hospitalized, and follow-up phone calls were conducted within 1 week of discharge, at 30 days, and at 90 days. At 30 and 90 days postdischarge, healthcare utilization was ascertained by review of medical records and patient report. Clinical data about the index hospitalization were also abstracted. The Vanderbilt University Institutional Review Board approved the study.

Study Population

Patients hospitalized from 2011 to 2015 with a likely diagnosis of acute coronary syndrome and/or ADHF, as determined by a physician’s review of the medical record, were identified as potentially eligible. Research assistants assessed these patients for the presence of the following exclusion criteria: less than 18 years of age, non-English speaking, unstable psychiatric illness, a low likelihood of follow-up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. Additionally, those with severe cognitive impairment, as assessed from the medical record (such as seeing a note describing dementia), and those with delirium, as assessed by the brief confusion assessment method, were excluded from enrollment in the study.^18,19 Those who died before discharge or during the 30-day follow-up period were excluded. For this analysis, we restricted our sample to only include participants who were hospitalized for ADHF.

 

Outcome Measure: 30-Day Readmission

The main outcome was all-cause readmission to any hospital within 30 days of discharge, as determined by patient interview, review of electronic medical records from VUMC, and review of outside hospital records.

Main Exposures: Numeracy, Health Literacy, and Cognitive Impairment

Numeracy was assessed with a 3-item version of the Subjective Numeracy Scale (SNS-3), which quantifies the patients perceived quantitative abilities.²⁰ Other authors have shown that the SNS-3 has a correlation coefficient of 0.88 with the full-length SNS-8 and a Cronbach’s alpha of 0.78.^20-22 The SNS-3 is reported as the mean on a scale from 1 to 6, with higher scores reflecting higher numeracy.

Subjective health literacy was assessed by using the 3-item Brief Health Literacy Screen (BHLS).²³ Scores range from 3 to 15, with higher scores reflecting higher literacy. Objective health literacy was assessed with the short form of the Test of Functional Health Literacy in Adults (sTOFHLA).^24,25 Scores may be categorized as inadequate (0-16), marginal (17-22), or adequate (23-36).

We assessed cognition by using the 10-item Short Portable Mental Status Questionnaire (SPMSQ).²⁶ The SPMSQ, which describes a person’s capacity for memory, structured thought, and orientation, has been validated and has demonstrated good reliability and validity.²⁷ Scores of 0 were considered to reflect intact cognition, and scores of 1 or more were considered to reflect any cognitive impairment, a scoring approach employed by other authors.²⁸ We used this approach, rather than the traditional scoring system developed by Pfeiffer et al.²⁶ (1975), because it would be the most sensitive to detect any cognitive impairment in the VICS cohort, which excluded those with severe cognition impairment, dementia, and delirium.

Covariates

During the hospitalization, participants completed an in-person interviewer-administered baseline assessment composed of demographic information, including age, self-reported race (white and nonwhite), educational attainment, home status (married, not married and living with someone, not married and living alone), and household income.

Clinical and diagnostic characteristics abstracted from the medical record included a medical history of HF, HF subtype (classified by left ventricular ejection fraction [LVEF]), coronary artery disease, chronic obstructive pulmonary disease (COPD), diabetes mellitus (DM), and comorbidity burden as summarized by the van Walraven-Elixhauser score.^29,30 Depressive symptoms were assessed during the 2 weeks prior to the hospitalization by using the first 8 items of the Patient Health Questionnaire.³¹ Scores ranged from 0 to 24, with higher scores reflecting more severe depressive symptoms. Laboratory values included estimated glomerular filtration rate (eGFR), hemoglobin (g/dl), sodium (mg/L), and brain natriuretic peptide (BNP) (pg/ml) from the last laboratory draw before discharge. Smoking status was also assessed (current and former/nonsmokers).

Hospitalization characteristics included length of stay in days, number of prior admissions in the last year, and transfer to the intensive care unit during the index admission.

Statistical Analysis

Descriptive statistics were used to summarize patient characteristics. The Kruskal-Wallis test and the Pearson χ² test were used to determine the association between patient characteristics and levels of numeracy, literacy, and cognition separately. The unadjusted relationship between patient characteristics and 30-day readmission was assessed by using Wilcoxon rank sums tests for continuous variables and Pearson χ² tests for categorical variables. In addition, a correlation matrix was performed to assess the correlations between numeracy, health literacy, and cognition (supplementary Figure 1).

To examine the association between numeracy, health literacy, and cognition and 30-day readmissions, a series of multivariable Poisson (log-linear) regression models were fit.³² Like other studies, numeracy, health literacy, and cognition were examined as categorical and continuous measures in models.³³ Each model was modified with a sandwich estimator for robust standard errors. Log-linear models were chosen over logistic regression models for ease of interpretation because (exponentiated) parameters correspond to risk ratios (RRs) as opposed to odds ratios. Furthermore, the fitting challenges associated with log-linear models when predicted probabilities are near 0 or 1 were not present in these analyses. Redundancy analyses were conducted to ensure that independent variables were not highly correlated with a linear combination of the other independent variables. To avoid case-wise deletion of records with missing covariates, we employed multiple imputation with 10 imputation samples by using predictive mean matching.^34,35 All analyses were conducted in R version 3.1.2 (The R Foundation, Vienna, Austria).³⁶

RESULTS

Overall, 883 patients were included in this analysis (supplementary Figure 2). Of the 883 participants, 46% were female and 76% were white (Table 1). Their median age was 60 years (interdecile range [IDR] 39-78) and the median educational attainment was 13.5 years (IDR 11-18).

Characteristics of the study sample by levels of subjective numeracy, objective health literacy, and cognition are shown in Table 1. A total of 33.9% had inadequate health numeracy (SNS scores 1-3 on a scale of 1-6) with an overall mean subjective numeracy score of 4.3 (standard deviation ± 1.3). Patients with inadequate numeracy were more likely to be women, nonwhite, and have lower education and income. Overall, 24.6% of the study population had inadequate/marginal objective health literacy, which is similar to the 26.1% with inadequate health literacy by the subjective literacy scale (BHLS scores 3-9 on a scale of 3-15) (supplementary Table 1). Patients with inadequate objective health literacy were more likely to be older, nonwhite, have less education and income, and more comorbidities compared with those with marginal/adequate health literacy. Overall, 53% of participants had any cognitive impairment (SPMSQ score = 1 or greater). They were more likely to be older, female, have less education and income, a greater number of comorbidities, and a higher severity of HF during the index admission compared with those with intact cognition.

A total of 23.8% (n = 210) of patients were readmitted within 30 days of discharge (Table 2). There was no statistically significant difference in readmission by numeracy level (P = .66). Readmitted patients were more likely to have lower objective health literacy compared with those who were not readmitted (27.1 vs 28.3; P = .04). A higher percentage of readmitted patients were cognitively impaired (57%) compared with those not readmitted (51%); however, this difference was not statistically significant (P = .11). Readmitted patients did not differ from nonreadmitted patients by demographic factors (supplementary Table 2). They were, however, more likely to have a history of HF, COPD, diabetes, CKD, higher Elixhauser scores, lower eGFR and lower sodium prior to discharge, and a greater number of prior readmissions in the last 12 months compared with those who were not readmitted (all P < .05).

In unadjusted and adjusted analyses, no statistically significant associations were seen between numeracy and the risk of 30-day readmission (Table 3). Additionally, in the adjusted analyses, there was no statistically significant association between objective health literacy or cognition and 30-day readmission. (supplementary Table 3). In a fully adjusted model, a history of diabetes was associated with a 30% greater risk of 30-day readmission compared with patients without a history of diabetes (RR = 1.30; P = .04) (supplementary Table 3). Per a 13-point increase in the Elixhauser score, the risk of readmission within 30 days increased by approximately 21% (RR = 1.21; P = .02). Additionally, having 3 prior hospital admissions in the previous 12 months was associated with a 30% higher risk of readmission than having 2 or fewer prior hospital admissions (RR = 1.3; P < .001).

 

DISCUSSION

This is the first study to examine the effect of numeracy alongside literacy and cognition on 30-day readmission risk among patients hospitalized with ADHF. Overall, we found that 33.9% of participants had inadequate numeracy skills, and 24.6% had inadequate or marginal health literacy. In unadjusted and adjusted models, numeracy was not associated with 30-day readmission. Although (objective) low health literacy was associated with 30-day readmission in unadjusted models, it was not in adjusted models. Additionally, though 53% of participants had any cognitive impairment, readmission did not differ significantly by this factor. Taken together, these findings suggest that other factors may be greater determinants of 30-day readmissions among patients hospitalized for ADHF.

Only 1 other study has examined the effect of numeracy on readmission risk among patients hospitalized for HF. In this multicenter prospective study, McNaughton et al.³⁷ found low numeracy to be associated with higher odds of recidivism to the emergency department (ED) or hospital within 30 days. Our findings may differ from theirs for a few reasons. First, their study had a significantly higher percentage of individuals with low numeracy (55%) compared with ours (33.9%). This may be because they did not exclude individuals with severe cognitive impairment, and their patient population was of lower socioeconomic status (SES) than ours. Low SES is associated with higher 30-day readmissions among HF patients^1,10 throughout the literature, and low numeracy is associated with low SES in other diseases.^13,38,39 Finally, they studied recidivism, which was defined as any unplanned return to the ED or hospital within 30 days of the index ED visit for acute HF. We only focused on 30-day readmissions, which also may explain why our results differed.

We found that health literacy was not associated with 30-day readmissions, which is consistent with the literature. Although an association between health literacy and mortality exists among adults with HF, several studies have not found an association between health literacy and 30- and 90-day readmission among adults hospitalized for HF.^8,9,40 Although we found an association between objective health literacy and 30-day readmission in unadjusted analyses, we did not find one in the multivariable model. This, along with our numeracy finding, suggests that numeracy and literacy may not be driving the 30-day readmission risk among patients hospitalized with ADHF.

We examined cognition alongside numeracy and literacy because it is a prevalent condition among HF patients and because it is associated with adverse outcomes among patients with HF, including readmission.^41,42 Studies have shown that HF preferentially affects certain cognitive domains,⁴³ some of which are vital to HF self-care activities. We found that 53% of patients had any cognitive impairment, which is consistent with the literature of adults hospitalized for ADHF.^44,45 Cognitive impairment was not, however, associated with 30-day readmissions. There may be a couple reasons for this. First, we measured cognitive impairment with the SPMSQ, which, although widely used and well-validated, does not assess executive function, the domain most commonly affected in HF patients with cognitive impairment.⁴⁶ Second, patients with severe cognitive impairment and those with delirium were excluded from this study, which may have limited our ability to detect differences in readmission by this factor.

As in prior studies, we found that a history of DM and more hospitalizations in the prior year were independently associated with 30-day readmissions in fully adjusted models. Like other studies, in adjusted models, we found that LVEF and a history of HF were not independently associated with 30-day readmission.^47-49 This, however, is not surprising because recent studies have shown that, although HF patients are at risk for multiple hospitalizations, early readmission after a hospitalization for ADHF specifically is often because of reasons unrelated to HF or a non-cardiovascular cause in general.^50,51

Although a negative study, several important themes emerged. First, while we were able to assess numeracy, health literacy, and cognition, none of these measures were HF-specific. It is possible that we did not see an effect on readmission because our instruments failed to assess domains specific to HF, such as monitoring weight changes, following a low-salt diet, and interpreting blood pressure. Currently, however, no HF-specific objective numeracy measure exists. With respect to health literacy, only 1 HF-specific measure exists,⁵² although it was only recently developed and validated. Second, while numeracy may not be a driving influence of all-cause 30-day readmissions, it may be associated with other health behaviors and quality metrics that we did not examine here, such as self-care, medication adherence, and HF-specific readmissions. Third, it is likely that the progression of HF itself, as well as the clinical management of patients following discharge, contribute significantly to 30-day readmissions. Increased attention to predischarge processes for HF patients occurred at VUMC during the study period; close follow-up and evidence-directed therapies may have mitigated some of the expected associations. Finally, we were not able to assess numeracy of participants’ primary caregivers who may help patients at home, especially postdischarge. Though a number of studies have examined the role of family caregivers in the management of HF,^53,54 none have examined numeracy levels of caregivers in the context of HF, and this may be worth doing in future studies.

Overall, our study has several strengths. The size of the cohort is large and there were high response rates during the follow-up period. Unlike other HF readmission studies, VICS accounts for readmissions to outside hospitals. Approximately 35% of all hospitalizations in VICS are to outside facilities. Thus, the ascertainment of readmissions to hospitals other than Vanderbilt is more comprehensive than if readmissions to VUMC were only considered. We were able to include a number of clinical comorbidities, laboratory and diagnostic tests from the index admission, and hospitalization characteristics in our analyses. Finally, we performed additional analyses to investigate the correlation between numeracy, literacy, and cognition; ultimately, we found that the majority of these correlations were weak, which supports our ability to study them simultaneously among VICS participants.

Nonetheless, we note some limitations. Although we captured readmissions to outside hospitals, the study took place at a single referral center in Tennessee. Though patients were diverse in age and comorbidities, they were mostly white and of higher SES. Finally, we used home status as a proxy for social support, which may underestimate the support that home care workers provide.

In conclusion, in this prospective longitudinal study of adults hospitalized with ADHF, inadequate numeracy was present in more than a third of patients, and low health literacy was present in roughly a quarter of patients. Neither numeracy nor health literacy, however, were associated with 30-day readmissions in adjusted analyses. Any cognitive impairment, although present in roughly one-half of patients, was not associated with 30-day readmission either. Our findings suggest that other influences may play a more dominant role in determining 30-day readmission rates in patients hospitalized for ADHF than inadequate numeracy, low health literacy, or cognitive impairment as assessed here.

 

Acknowledgments

This research was supported by the National Heart, Lung, and Blood Institute (R01 HL109388) and in part by the National Center for Advancing Translational Sciences (UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health. The authors’ funding sources did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication. Dr. Sterling is supported by T32HS000066 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Mixon has a VA Health Services Research and Development Service Career Development Award at the Tennessee Valley Healthcare System, Department of Veterans Affairs (CDA 12-168). This material was presented at the Society of General Internal Medicine Annual Meeting on April 20, 2017, in Washington, DC.

Disclosure

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, all outside of the submitted work. Dr. Rothman and Dr. Wallston report personal fees from EdLogics outside of the submitted work. All of the other authors have nothing to disclose

Most studies to identify risk factors for readmission among patients with heart failure (HF) have focused on demographic and clinical characteristics.^1,2 Although easy to extract from administrative databases, this approach fails to capture the complex psychosocial and cognitive factors that influence the ability of HF patients to manage their disease in the postdischarge period, as depicted in the framework by Meyers et al.³ (2014). To date, studies have found low health literacy, decreased social support, and cognitive impairment to be associated with health behaviors and outcomes among HF patients, including decreased self-care,⁴ low HF-specific knowledge,⁵ medication nonadherence,⁶ hospitalizations,⁷ and mortality.^8-10 Less, however, is known about the effect of numeracy on HF outcomes, such as 30-day readmission.

Numeracy, or quantitative literacy, refers to the ability to access, understand, and apply numerical data to health-related decisions.¹¹ It is estimated that 110 million people in the United States have limited numeracy skills.¹² Low numeracy is a risk factor for poor glycemic control among patients with diabetes,¹³ medication adherence in HIV/AIDS,¹⁴ and worse blood pressure control in hypertensives.¹⁵ Much like these conditions, HF requires that patients understand, use, and act on numerical information. Maintaining a low-salt diet, monitoring weight, adjusting diuretic doses, and measuring blood pressure are tasks that HF patients are asked to perform on a daily or near-daily basis. These tasks are particularly important in the posthospitalization period and could be complicated by medication changes, which might create additional challenges for patients with inadequate numeracy. Additionally, cognitive impairment, which is a highly prevalent comorbid condition among adults with HF,^16,17 might impose additional barriers for those with inadequate numeracy who do not have adequate social support. However, to date, numeracy in the context of HF has not been well described.

Herein, we examined the effects of numeracy, alongside health literacy and cognition, on 30-day readmission risk among patients hospitalized for acute decompensated HF (ADHF).

METHODS

Study Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective observational study of patients admitted with cardiovascular disease to Vanderbilt University Medical Center (VUMC), an academic tertiary care hospital. VICS was designed to investigate the impact of social determinants of health on postdischarge health outcomes. A detailed description of the study rationale, design, and methods is described elsewhere.³

Briefly, participants completed a baseline interview while hospitalized, and follow-up phone calls were conducted within 1 week of discharge, at 30 days, and at 90 days. At 30 and 90 days postdischarge, healthcare utilization was ascertained by review of medical records and patient report. Clinical data about the index hospitalization were also abstracted. The Vanderbilt University Institutional Review Board approved the study.

Study Population

Patients hospitalized from 2011 to 2015 with a likely diagnosis of acute coronary syndrome and/or ADHF, as determined by a physician’s review of the medical record, were identified as potentially eligible. Research assistants assessed these patients for the presence of the following exclusion criteria: less than 18 years of age, non-English speaking, unstable psychiatric illness, a low likelihood of follow-up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. Additionally, those with severe cognitive impairment, as assessed from the medical record (such as seeing a note describing dementia), and those with delirium, as assessed by the brief confusion assessment method, were excluded from enrollment in the study.^18,19 Those who died before discharge or during the 30-day follow-up period were excluded. For this analysis, we restricted our sample to only include participants who were hospitalized for ADHF.

 

Outcome Measure: 30-Day Readmission

The main outcome was all-cause readmission to any hospital within 30 days of discharge, as determined by patient interview, review of electronic medical records from VUMC, and review of outside hospital records.

Main Exposures: Numeracy, Health Literacy, and Cognitive Impairment

Numeracy was assessed with a 3-item version of the Subjective Numeracy Scale (SNS-3), which quantifies the patients perceived quantitative abilities.²⁰ Other authors have shown that the SNS-3 has a correlation coefficient of 0.88 with the full-length SNS-8 and a Cronbach’s alpha of 0.78.^20-22 The SNS-3 is reported as the mean on a scale from 1 to 6, with higher scores reflecting higher numeracy.

Subjective health literacy was assessed by using the 3-item Brief Health Literacy Screen (BHLS).²³ Scores range from 3 to 15, with higher scores reflecting higher literacy. Objective health literacy was assessed with the short form of the Test of Functional Health Literacy in Adults (sTOFHLA).^24,25 Scores may be categorized as inadequate (0-16), marginal (17-22), or adequate (23-36).

We assessed cognition by using the 10-item Short Portable Mental Status Questionnaire (SPMSQ).²⁶ The SPMSQ, which describes a person’s capacity for memory, structured thought, and orientation, has been validated and has demonstrated good reliability and validity.²⁷ Scores of 0 were considered to reflect intact cognition, and scores of 1 or more were considered to reflect any cognitive impairment, a scoring approach employed by other authors.²⁸ We used this approach, rather than the traditional scoring system developed by Pfeiffer et al.²⁶ (1975), because it would be the most sensitive to detect any cognitive impairment in the VICS cohort, which excluded those with severe cognition impairment, dementia, and delirium.

Covariates

During the hospitalization, participants completed an in-person interviewer-administered baseline assessment composed of demographic information, including age, self-reported race (white and nonwhite), educational attainment, home status (married, not married and living with someone, not married and living alone), and household income.

Clinical and diagnostic characteristics abstracted from the medical record included a medical history of HF, HF subtype (classified by left ventricular ejection fraction [LVEF]), coronary artery disease, chronic obstructive pulmonary disease (COPD), diabetes mellitus (DM), and comorbidity burden as summarized by the van Walraven-Elixhauser score.^29,30 Depressive symptoms were assessed during the 2 weeks prior to the hospitalization by using the first 8 items of the Patient Health Questionnaire.³¹ Scores ranged from 0 to 24, with higher scores reflecting more severe depressive symptoms. Laboratory values included estimated glomerular filtration rate (eGFR), hemoglobin (g/dl), sodium (mg/L), and brain natriuretic peptide (BNP) (pg/ml) from the last laboratory draw before discharge. Smoking status was also assessed (current and former/nonsmokers).

Hospitalization characteristics included length of stay in days, number of prior admissions in the last year, and transfer to the intensive care unit during the index admission.

Statistical Analysis

Descriptive statistics were used to summarize patient characteristics. The Kruskal-Wallis test and the Pearson χ² test were used to determine the association between patient characteristics and levels of numeracy, literacy, and cognition separately. The unadjusted relationship between patient characteristics and 30-day readmission was assessed by using Wilcoxon rank sums tests for continuous variables and Pearson χ² tests for categorical variables. In addition, a correlation matrix was performed to assess the correlations between numeracy, health literacy, and cognition (supplementary Figure 1).

To examine the association between numeracy, health literacy, and cognition and 30-day readmissions, a series of multivariable Poisson (log-linear) regression models were fit.³² Like other studies, numeracy, health literacy, and cognition were examined as categorical and continuous measures in models.³³ Each model was modified with a sandwich estimator for robust standard errors. Log-linear models were chosen over logistic regression models for ease of interpretation because (exponentiated) parameters correspond to risk ratios (RRs) as opposed to odds ratios. Furthermore, the fitting challenges associated with log-linear models when predicted probabilities are near 0 or 1 were not present in these analyses. Redundancy analyses were conducted to ensure that independent variables were not highly correlated with a linear combination of the other independent variables. To avoid case-wise deletion of records with missing covariates, we employed multiple imputation with 10 imputation samples by using predictive mean matching.^34,35 All analyses were conducted in R version 3.1.2 (The R Foundation, Vienna, Austria).³⁶

RESULTS

Overall, 883 patients were included in this analysis (supplementary Figure 2). Of the 883 participants, 46% were female and 76% were white (Table 1). Their median age was 60 years (interdecile range [IDR] 39-78) and the median educational attainment was 13.5 years (IDR 11-18).

Characteristics of the study sample by levels of subjective numeracy, objective health literacy, and cognition are shown in Table 1. A total of 33.9% had inadequate health numeracy (SNS scores 1-3 on a scale of 1-6) with an overall mean subjective numeracy score of 4.3 (standard deviation ± 1.3). Patients with inadequate numeracy were more likely to be women, nonwhite, and have lower education and income. Overall, 24.6% of the study population had inadequate/marginal objective health literacy, which is similar to the 26.1% with inadequate health literacy by the subjective literacy scale (BHLS scores 3-9 on a scale of 3-15) (supplementary Table 1). Patients with inadequate objective health literacy were more likely to be older, nonwhite, have less education and income, and more comorbidities compared with those with marginal/adequate health literacy. Overall, 53% of participants had any cognitive impairment (SPMSQ score = 1 or greater). They were more likely to be older, female, have less education and income, a greater number of comorbidities, and a higher severity of HF during the index admission compared with those with intact cognition.

A total of 23.8% (n = 210) of patients were readmitted within 30 days of discharge (Table 2). There was no statistically significant difference in readmission by numeracy level (P = .66). Readmitted patients were more likely to have lower objective health literacy compared with those who were not readmitted (27.1 vs 28.3; P = .04). A higher percentage of readmitted patients were cognitively impaired (57%) compared with those not readmitted (51%); however, this difference was not statistically significant (P = .11). Readmitted patients did not differ from nonreadmitted patients by demographic factors (supplementary Table 2). They were, however, more likely to have a history of HF, COPD, diabetes, CKD, higher Elixhauser scores, lower eGFR and lower sodium prior to discharge, and a greater number of prior readmissions in the last 12 months compared with those who were not readmitted (all P < .05).

In unadjusted and adjusted analyses, no statistically significant associations were seen between numeracy and the risk of 30-day readmission (Table 3). Additionally, in the adjusted analyses, there was no statistically significant association between objective health literacy or cognition and 30-day readmission. (supplementary Table 3). In a fully adjusted model, a history of diabetes was associated with a 30% greater risk of 30-day readmission compared with patients without a history of diabetes (RR = 1.30; P = .04) (supplementary Table 3). Per a 13-point increase in the Elixhauser score, the risk of readmission within 30 days increased by approximately 21% (RR = 1.21; P = .02). Additionally, having 3 prior hospital admissions in the previous 12 months was associated with a 30% higher risk of readmission than having 2 or fewer prior hospital admissions (RR = 1.3; P < .001).

 

DISCUSSION

This is the first study to examine the effect of numeracy alongside literacy and cognition on 30-day readmission risk among patients hospitalized with ADHF. Overall, we found that 33.9% of participants had inadequate numeracy skills, and 24.6% had inadequate or marginal health literacy. In unadjusted and adjusted models, numeracy was not associated with 30-day readmission. Although (objective) low health literacy was associated with 30-day readmission in unadjusted models, it was not in adjusted models. Additionally, though 53% of participants had any cognitive impairment, readmission did not differ significantly by this factor. Taken together, these findings suggest that other factors may be greater determinants of 30-day readmissions among patients hospitalized for ADHF.

Only 1 other study has examined the effect of numeracy on readmission risk among patients hospitalized for HF. In this multicenter prospective study, McNaughton et al.³⁷ found low numeracy to be associated with higher odds of recidivism to the emergency department (ED) or hospital within 30 days. Our findings may differ from theirs for a few reasons. First, their study had a significantly higher percentage of individuals with low numeracy (55%) compared with ours (33.9%). This may be because they did not exclude individuals with severe cognitive impairment, and their patient population was of lower socioeconomic status (SES) than ours. Low SES is associated with higher 30-day readmissions among HF patients^1,10 throughout the literature, and low numeracy is associated with low SES in other diseases.^13,38,39 Finally, they studied recidivism, which was defined as any unplanned return to the ED or hospital within 30 days of the index ED visit for acute HF. We only focused on 30-day readmissions, which also may explain why our results differed.

We found that health literacy was not associated with 30-day readmissions, which is consistent with the literature. Although an association between health literacy and mortality exists among adults with HF, several studies have not found an association between health literacy and 30- and 90-day readmission among adults hospitalized for HF.^8,9,40 Although we found an association between objective health literacy and 30-day readmission in unadjusted analyses, we did not find one in the multivariable model. This, along with our numeracy finding, suggests that numeracy and literacy may not be driving the 30-day readmission risk among patients hospitalized with ADHF.

We examined cognition alongside numeracy and literacy because it is a prevalent condition among HF patients and because it is associated with adverse outcomes among patients with HF, including readmission.^41,42 Studies have shown that HF preferentially affects certain cognitive domains,⁴³ some of which are vital to HF self-care activities. We found that 53% of patients had any cognitive impairment, which is consistent with the literature of adults hospitalized for ADHF.^44,45 Cognitive impairment was not, however, associated with 30-day readmissions. There may be a couple reasons for this. First, we measured cognitive impairment with the SPMSQ, which, although widely used and well-validated, does not assess executive function, the domain most commonly affected in HF patients with cognitive impairment.⁴⁶ Second, patients with severe cognitive impairment and those with delirium were excluded from this study, which may have limited our ability to detect differences in readmission by this factor.

As in prior studies, we found that a history of DM and more hospitalizations in the prior year were independently associated with 30-day readmissions in fully adjusted models. Like other studies, in adjusted models, we found that LVEF and a history of HF were not independently associated with 30-day readmission.^47-49 This, however, is not surprising because recent studies have shown that, although HF patients are at risk for multiple hospitalizations, early readmission after a hospitalization for ADHF specifically is often because of reasons unrelated to HF or a non-cardiovascular cause in general.^50,51

Although a negative study, several important themes emerged. First, while we were able to assess numeracy, health literacy, and cognition, none of these measures were HF-specific. It is possible that we did not see an effect on readmission because our instruments failed to assess domains specific to HF, such as monitoring weight changes, following a low-salt diet, and interpreting blood pressure. Currently, however, no HF-specific objective numeracy measure exists. With respect to health literacy, only 1 HF-specific measure exists,⁵² although it was only recently developed and validated. Second, while numeracy may not be a driving influence of all-cause 30-day readmissions, it may be associated with other health behaviors and quality metrics that we did not examine here, such as self-care, medication adherence, and HF-specific readmissions. Third, it is likely that the progression of HF itself, as well as the clinical management of patients following discharge, contribute significantly to 30-day readmissions. Increased attention to predischarge processes for HF patients occurred at VUMC during the study period; close follow-up and evidence-directed therapies may have mitigated some of the expected associations. Finally, we were not able to assess numeracy of participants’ primary caregivers who may help patients at home, especially postdischarge. Though a number of studies have examined the role of family caregivers in the management of HF,^53,54 none have examined numeracy levels of caregivers in the context of HF, and this may be worth doing in future studies.

Overall, our study has several strengths. The size of the cohort is large and there were high response rates during the follow-up period. Unlike other HF readmission studies, VICS accounts for readmissions to outside hospitals. Approximately 35% of all hospitalizations in VICS are to outside facilities. Thus, the ascertainment of readmissions to hospitals other than Vanderbilt is more comprehensive than if readmissions to VUMC were only considered. We were able to include a number of clinical comorbidities, laboratory and diagnostic tests from the index admission, and hospitalization characteristics in our analyses. Finally, we performed additional analyses to investigate the correlation between numeracy, literacy, and cognition; ultimately, we found that the majority of these correlations were weak, which supports our ability to study them simultaneously among VICS participants.

Nonetheless, we note some limitations. Although we captured readmissions to outside hospitals, the study took place at a single referral center in Tennessee. Though patients were diverse in age and comorbidities, they were mostly white and of higher SES. Finally, we used home status as a proxy for social support, which may underestimate the support that home care workers provide.

In conclusion, in this prospective longitudinal study of adults hospitalized with ADHF, inadequate numeracy was present in more than a third of patients, and low health literacy was present in roughly a quarter of patients. Neither numeracy nor health literacy, however, were associated with 30-day readmissions in adjusted analyses. Any cognitive impairment, although present in roughly one-half of patients, was not associated with 30-day readmission either. Our findings suggest that other influences may play a more dominant role in determining 30-day readmission rates in patients hospitalized for ADHF than inadequate numeracy, low health literacy, or cognitive impairment as assessed here.

 

Acknowledgments

This research was supported by the National Heart, Lung, and Blood Institute (R01 HL109388) and in part by the National Center for Advancing Translational Sciences (UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health. The authors’ funding sources did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication. Dr. Sterling is supported by T32HS000066 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Mixon has a VA Health Services Research and Development Service Career Development Award at the Tennessee Valley Healthcare System, Department of Veterans Affairs (CDA 12-168). This material was presented at the Society of General Internal Medicine Annual Meeting on April 20, 2017, in Washington, DC.

Disclosure

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, all outside of the submitted work. Dr. Rothman and Dr. Wallston report personal fees from EdLogics outside of the submitted work. All of the other authors have nothing to disclose

Most studies to identify risk factors for readmission among patients with heart failure (HF) have focused on demographic and clinical characteristics.^1,2 Although easy to extract from administrative databases, this approach fails to capture the complex psychosocial and cognitive factors that influence the ability of HF patients to manage their disease in the postdischarge period, as depicted in the framework by Meyers et al.³ (2014). To date, studies have found low health literacy, decreased social support, and cognitive impairment to be associated with health behaviors and outcomes among HF patients, including decreased self-care,⁴ low HF-specific knowledge,⁵ medication nonadherence,⁶ hospitalizations,⁷ and mortality.^8-10 Less, however, is known about the effect of numeracy on HF outcomes, such as 30-day readmission.

Numeracy, or quantitative literacy, refers to the ability to access, understand, and apply numerical data to health-related decisions.¹¹ It is estimated that 110 million people in the United States have limited numeracy skills.¹² Low numeracy is a risk factor for poor glycemic control among patients with diabetes,¹³ medication adherence in HIV/AIDS,¹⁴ and worse blood pressure control in hypertensives.¹⁵ Much like these conditions, HF requires that patients understand, use, and act on numerical information. Maintaining a low-salt diet, monitoring weight, adjusting diuretic doses, and measuring blood pressure are tasks that HF patients are asked to perform on a daily or near-daily basis. These tasks are particularly important in the posthospitalization period and could be complicated by medication changes, which might create additional challenges for patients with inadequate numeracy. Additionally, cognitive impairment, which is a highly prevalent comorbid condition among adults with HF,^16,17 might impose additional barriers for those with inadequate numeracy who do not have adequate social support. However, to date, numeracy in the context of HF has not been well described.

Herein, we examined the effects of numeracy, alongside health literacy and cognition, on 30-day readmission risk among patients hospitalized for acute decompensated HF (ADHF).

METHODS

Study Design

The Vanderbilt Inpatient Cohort Study (VICS) is a prospective observational study of patients admitted with cardiovascular disease to Vanderbilt University Medical Center (VUMC), an academic tertiary care hospital. VICS was designed to investigate the impact of social determinants of health on postdischarge health outcomes. A detailed description of the study rationale, design, and methods is described elsewhere.³

Briefly, participants completed a baseline interview while hospitalized, and follow-up phone calls were conducted within 1 week of discharge, at 30 days, and at 90 days. At 30 and 90 days postdischarge, healthcare utilization was ascertained by review of medical records and patient report. Clinical data about the index hospitalization were also abstracted. The Vanderbilt University Institutional Review Board approved the study.

Study Population

Patients hospitalized from 2011 to 2015 with a likely diagnosis of acute coronary syndrome and/or ADHF, as determined by a physician’s review of the medical record, were identified as potentially eligible. Research assistants assessed these patients for the presence of the following exclusion criteria: less than 18 years of age, non-English speaking, unstable psychiatric illness, a low likelihood of follow-up (eg, no reliable telephone number), on hospice, or otherwise too ill to complete an interview. Additionally, those with severe cognitive impairment, as assessed from the medical record (such as seeing a note describing dementia), and those with delirium, as assessed by the brief confusion assessment method, were excluded from enrollment in the study.^18,19 Those who died before discharge or during the 30-day follow-up period were excluded. For this analysis, we restricted our sample to only include participants who were hospitalized for ADHF.

 

Outcome Measure: 30-Day Readmission

The main outcome was all-cause readmission to any hospital within 30 days of discharge, as determined by patient interview, review of electronic medical records from VUMC, and review of outside hospital records.

Main Exposures: Numeracy, Health Literacy, and Cognitive Impairment

Numeracy was assessed with a 3-item version of the Subjective Numeracy Scale (SNS-3), which quantifies the patients perceived quantitative abilities.²⁰ Other authors have shown that the SNS-3 has a correlation coefficient of 0.88 with the full-length SNS-8 and a Cronbach’s alpha of 0.78.^20-22 The SNS-3 is reported as the mean on a scale from 1 to 6, with higher scores reflecting higher numeracy.

Subjective health literacy was assessed by using the 3-item Brief Health Literacy Screen (BHLS).²³ Scores range from 3 to 15, with higher scores reflecting higher literacy. Objective health literacy was assessed with the short form of the Test of Functional Health Literacy in Adults (sTOFHLA).^24,25 Scores may be categorized as inadequate (0-16), marginal (17-22), or adequate (23-36).

We assessed cognition by using the 10-item Short Portable Mental Status Questionnaire (SPMSQ).²⁶ The SPMSQ, which describes a person’s capacity for memory, structured thought, and orientation, has been validated and has demonstrated good reliability and validity.²⁷ Scores of 0 were considered to reflect intact cognition, and scores of 1 or more were considered to reflect any cognitive impairment, a scoring approach employed by other authors.²⁸ We used this approach, rather than the traditional scoring system developed by Pfeiffer et al.²⁶ (1975), because it would be the most sensitive to detect any cognitive impairment in the VICS cohort, which excluded those with severe cognition impairment, dementia, and delirium.

Covariates

During the hospitalization, participants completed an in-person interviewer-administered baseline assessment composed of demographic information, including age, self-reported race (white and nonwhite), educational attainment, home status (married, not married and living with someone, not married and living alone), and household income.

Clinical and diagnostic characteristics abstracted from the medical record included a medical history of HF, HF subtype (classified by left ventricular ejection fraction [LVEF]), coronary artery disease, chronic obstructive pulmonary disease (COPD), diabetes mellitus (DM), and comorbidity burden as summarized by the van Walraven-Elixhauser score.^29,30 Depressive symptoms were assessed during the 2 weeks prior to the hospitalization by using the first 8 items of the Patient Health Questionnaire.³¹ Scores ranged from 0 to 24, with higher scores reflecting more severe depressive symptoms. Laboratory values included estimated glomerular filtration rate (eGFR), hemoglobin (g/dl), sodium (mg/L), and brain natriuretic peptide (BNP) (pg/ml) from the last laboratory draw before discharge. Smoking status was also assessed (current and former/nonsmokers).

Hospitalization characteristics included length of stay in days, number of prior admissions in the last year, and transfer to the intensive care unit during the index admission.

Statistical Analysis

Descriptive statistics were used to summarize patient characteristics. The Kruskal-Wallis test and the Pearson χ² test were used to determine the association between patient characteristics and levels of numeracy, literacy, and cognition separately. The unadjusted relationship between patient characteristics and 30-day readmission was assessed by using Wilcoxon rank sums tests for continuous variables and Pearson χ² tests for categorical variables. In addition, a correlation matrix was performed to assess the correlations between numeracy, health literacy, and cognition (supplementary Figure 1).

To examine the association between numeracy, health literacy, and cognition and 30-day readmissions, a series of multivariable Poisson (log-linear) regression models were fit.³² Like other studies, numeracy, health literacy, and cognition were examined as categorical and continuous measures in models.³³ Each model was modified with a sandwich estimator for robust standard errors. Log-linear models were chosen over logistic regression models for ease of interpretation because (exponentiated) parameters correspond to risk ratios (RRs) as opposed to odds ratios. Furthermore, the fitting challenges associated with log-linear models when predicted probabilities are near 0 or 1 were not present in these analyses. Redundancy analyses were conducted to ensure that independent variables were not highly correlated with a linear combination of the other independent variables. To avoid case-wise deletion of records with missing covariates, we employed multiple imputation with 10 imputation samples by using predictive mean matching.^34,35 All analyses were conducted in R version 3.1.2 (The R Foundation, Vienna, Austria).³⁶

RESULTS

Overall, 883 patients were included in this analysis (supplementary Figure 2). Of the 883 participants, 46% were female and 76% were white (Table 1). Their median age was 60 years (interdecile range [IDR] 39-78) and the median educational attainment was 13.5 years (IDR 11-18).

Characteristics of the study sample by levels of subjective numeracy, objective health literacy, and cognition are shown in Table 1. A total of 33.9% had inadequate health numeracy (SNS scores 1-3 on a scale of 1-6) with an overall mean subjective numeracy score of 4.3 (standard deviation ± 1.3). Patients with inadequate numeracy were more likely to be women, nonwhite, and have lower education and income. Overall, 24.6% of the study population had inadequate/marginal objective health literacy, which is similar to the 26.1% with inadequate health literacy by the subjective literacy scale (BHLS scores 3-9 on a scale of 3-15) (supplementary Table 1). Patients with inadequate objective health literacy were more likely to be older, nonwhite, have less education and income, and more comorbidities compared with those with marginal/adequate health literacy. Overall, 53% of participants had any cognitive impairment (SPMSQ score = 1 or greater). They were more likely to be older, female, have less education and income, a greater number of comorbidities, and a higher severity of HF during the index admission compared with those with intact cognition.

A total of 23.8% (n = 210) of patients were readmitted within 30 days of discharge (Table 2). There was no statistically significant difference in readmission by numeracy level (P = .66). Readmitted patients were more likely to have lower objective health literacy compared with those who were not readmitted (27.1 vs 28.3; P = .04). A higher percentage of readmitted patients were cognitively impaired (57%) compared with those not readmitted (51%); however, this difference was not statistically significant (P = .11). Readmitted patients did not differ from nonreadmitted patients by demographic factors (supplementary Table 2). They were, however, more likely to have a history of HF, COPD, diabetes, CKD, higher Elixhauser scores, lower eGFR and lower sodium prior to discharge, and a greater number of prior readmissions in the last 12 months compared with those who were not readmitted (all P < .05).

In unadjusted and adjusted analyses, no statistically significant associations were seen between numeracy and the risk of 30-day readmission (Table 3). Additionally, in the adjusted analyses, there was no statistically significant association between objective health literacy or cognition and 30-day readmission. (supplementary Table 3). In a fully adjusted model, a history of diabetes was associated with a 30% greater risk of 30-day readmission compared with patients without a history of diabetes (RR = 1.30; P = .04) (supplementary Table 3). Per a 13-point increase in the Elixhauser score, the risk of readmission within 30 days increased by approximately 21% (RR = 1.21; P = .02). Additionally, having 3 prior hospital admissions in the previous 12 months was associated with a 30% higher risk of readmission than having 2 or fewer prior hospital admissions (RR = 1.3; P < .001).

 

DISCUSSION

This is the first study to examine the effect of numeracy alongside literacy and cognition on 30-day readmission risk among patients hospitalized with ADHF. Overall, we found that 33.9% of participants had inadequate numeracy skills, and 24.6% had inadequate or marginal health literacy. In unadjusted and adjusted models, numeracy was not associated with 30-day readmission. Although (objective) low health literacy was associated with 30-day readmission in unadjusted models, it was not in adjusted models. Additionally, though 53% of participants had any cognitive impairment, readmission did not differ significantly by this factor. Taken together, these findings suggest that other factors may be greater determinants of 30-day readmissions among patients hospitalized for ADHF.

Only 1 other study has examined the effect of numeracy on readmission risk among patients hospitalized for HF. In this multicenter prospective study, McNaughton et al.³⁷ found low numeracy to be associated with higher odds of recidivism to the emergency department (ED) or hospital within 30 days. Our findings may differ from theirs for a few reasons. First, their study had a significantly higher percentage of individuals with low numeracy (55%) compared with ours (33.9%). This may be because they did not exclude individuals with severe cognitive impairment, and their patient population was of lower socioeconomic status (SES) than ours. Low SES is associated with higher 30-day readmissions among HF patients^1,10 throughout the literature, and low numeracy is associated with low SES in other diseases.^13,38,39 Finally, they studied recidivism, which was defined as any unplanned return to the ED or hospital within 30 days of the index ED visit for acute HF. We only focused on 30-day readmissions, which also may explain why our results differed.

We found that health literacy was not associated with 30-day readmissions, which is consistent with the literature. Although an association between health literacy and mortality exists among adults with HF, several studies have not found an association between health literacy and 30- and 90-day readmission among adults hospitalized for HF.^8,9,40 Although we found an association between objective health literacy and 30-day readmission in unadjusted analyses, we did not find one in the multivariable model. This, along with our numeracy finding, suggests that numeracy and literacy may not be driving the 30-day readmission risk among patients hospitalized with ADHF.

We examined cognition alongside numeracy and literacy because it is a prevalent condition among HF patients and because it is associated with adverse outcomes among patients with HF, including readmission.^41,42 Studies have shown that HF preferentially affects certain cognitive domains,⁴³ some of which are vital to HF self-care activities. We found that 53% of patients had any cognitive impairment, which is consistent with the literature of adults hospitalized for ADHF.^44,45 Cognitive impairment was not, however, associated with 30-day readmissions. There may be a couple reasons for this. First, we measured cognitive impairment with the SPMSQ, which, although widely used and well-validated, does not assess executive function, the domain most commonly affected in HF patients with cognitive impairment.⁴⁶ Second, patients with severe cognitive impairment and those with delirium were excluded from this study, which may have limited our ability to detect differences in readmission by this factor.

As in prior studies, we found that a history of DM and more hospitalizations in the prior year were independently associated with 30-day readmissions in fully adjusted models. Like other studies, in adjusted models, we found that LVEF and a history of HF were not independently associated with 30-day readmission.^47-49 This, however, is not surprising because recent studies have shown that, although HF patients are at risk for multiple hospitalizations, early readmission after a hospitalization for ADHF specifically is often because of reasons unrelated to HF or a non-cardiovascular cause in general.^50,51

Although a negative study, several important themes emerged. First, while we were able to assess numeracy, health literacy, and cognition, none of these measures were HF-specific. It is possible that we did not see an effect on readmission because our instruments failed to assess domains specific to HF, such as monitoring weight changes, following a low-salt diet, and interpreting blood pressure. Currently, however, no HF-specific objective numeracy measure exists. With respect to health literacy, only 1 HF-specific measure exists,⁵² although it was only recently developed and validated. Second, while numeracy may not be a driving influence of all-cause 30-day readmissions, it may be associated with other health behaviors and quality metrics that we did not examine here, such as self-care, medication adherence, and HF-specific readmissions. Third, it is likely that the progression of HF itself, as well as the clinical management of patients following discharge, contribute significantly to 30-day readmissions. Increased attention to predischarge processes for HF patients occurred at VUMC during the study period; close follow-up and evidence-directed therapies may have mitigated some of the expected associations. Finally, we were not able to assess numeracy of participants’ primary caregivers who may help patients at home, especially postdischarge. Though a number of studies have examined the role of family caregivers in the management of HF,^53,54 none have examined numeracy levels of caregivers in the context of HF, and this may be worth doing in future studies.

Overall, our study has several strengths. The size of the cohort is large and there were high response rates during the follow-up period. Unlike other HF readmission studies, VICS accounts for readmissions to outside hospitals. Approximately 35% of all hospitalizations in VICS are to outside facilities. Thus, the ascertainment of readmissions to hospitals other than Vanderbilt is more comprehensive than if readmissions to VUMC were only considered. We were able to include a number of clinical comorbidities, laboratory and diagnostic tests from the index admission, and hospitalization characteristics in our analyses. Finally, we performed additional analyses to investigate the correlation between numeracy, literacy, and cognition; ultimately, we found that the majority of these correlations were weak, which supports our ability to study them simultaneously among VICS participants.

Nonetheless, we note some limitations. Although we captured readmissions to outside hospitals, the study took place at a single referral center in Tennessee. Though patients were diverse in age and comorbidities, they were mostly white and of higher SES. Finally, we used home status as a proxy for social support, which may underestimate the support that home care workers provide.

In conclusion, in this prospective longitudinal study of adults hospitalized with ADHF, inadequate numeracy was present in more than a third of patients, and low health literacy was present in roughly a quarter of patients. Neither numeracy nor health literacy, however, were associated with 30-day readmissions in adjusted analyses. Any cognitive impairment, although present in roughly one-half of patients, was not associated with 30-day readmission either. Our findings suggest that other influences may play a more dominant role in determining 30-day readmission rates in patients hospitalized for ADHF than inadequate numeracy, low health literacy, or cognitive impairment as assessed here.

 

Acknowledgments

This research was supported by the National Heart, Lung, and Blood Institute (R01 HL109388) and in part by the National Center for Advancing Translational Sciences (UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the National Institutes of Health. The authors’ funding sources did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication. Dr. Sterling is supported by T32HS000066 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. Dr. Mixon has a VA Health Services Research and Development Service Career Development Award at the Tennessee Valley Healthcare System, Department of Veterans Affairs (CDA 12-168). This material was presented at the Society of General Internal Medicine Annual Meeting on April 20, 2017, in Washington, DC.

Disclosure

Dr. Kripalani reports personal fees from Verustat, personal fees from SAI Interactive, and equity from Bioscape Digital, all outside of the submitted work. Dr. Rothman and Dr. Wallston report personal fees from EdLogics outside of the submitted work. All of the other authors have nothing to disclose

A special concern since the institution of hospital readmission penalties¹ is the transitions in care of a patient from one care setting to another, often at hospital discharge. Burke et al.² proposed a framework for an ideal transition in care (ITC) to study and improve transitions from the hospital to home. The features in the ITC were identified based upon their inclusion in the interventions that improved discharge outcomes.^3-5 Inspired by the ITC and other patient risk tools,⁶ we identified 10 domains of transitional care needs ([TCN] specified below), which we define as patient-centered risk factors that should be addressed to foster a safe and effective transition in care.⁷

One particularly important risk factor in patient self-management at transition points is health literacy, a patient’s ability to obtain, understand, and use basic health information and services. Low health literacy affects approximately 26% to 36% of adults in the United States.^8,9 Health literacy is associated with many factors that may affect successful navigation of care transitions, including doctor-patient communication,^10,11 understanding of the medication regimen,¹² and self-management.^13-15 Research has also demonstrated an association between low health literacy and poor outcomes after hospital discharge, including medication errors,¹⁶ 30-day hospital readmission,¹⁷ and mortality.¹⁸ Transitional care initiatives have begun to incorporate health literacy into patient risk assessments⁶ and provide specific attention to low health literacy in interventions to reduce adverse drug events and readmission.^4,19 Training programs for medical students and nurses advise teaching skills in health literacy as part of fostering effective transitions in care.^20,21

Although low health literacy is generally recognized as a barrier to patient education and self-management, little is known about whether patients with low health literacy are more likely to have other risk factors that could further increase their risk for poor transitions in care. A better understanding of associated risks would inform and improve patient care. We hypothesized that TCNs are more common among patients with low health literacy, as compared with those with adequate health literacy. We also aimed to describe the relationship between low health literacy and specific TCNs in order to guide clinical care and future interventions.

METHODS

Setting

The present study is a cross-sectional analysis of data from a quality improvement (QI) intervention that was performed at Vanderbilt University Medical Center, a tertiary care facility in Nashville, Tennessee. The QI intervention, My Health Team (MHT), was funded by the Centers for Medicare and Medicaid Services Innovation Award program. The overall MHT program included outpatient care coordination for chronic disease management as well as a transitional care program that was designed to reduce hospital readmission. The latter included an inpatient needs assessment (which provided data for the present analysis), inpatient intervention, and postdischarge phone follow-up. The MHT initiative was reviewed by the institutional review board (IRB), which deemed it a QI program and granted a waiver of informed consent. The present secondary data analysis was reviewed and approved by the IRB.

Sample

Patients were identified for inclusion in the MHT transitions of care program if the presenting problem for hospital admission was pneumonia, chronic obstructive pulmonary disease (COPD) exacerbation, or decompensated heart failure, as determined by the review of clinical documentation by nurse transition care coordinators (TCCs). Adults over the age of 18 years were eligible, though priority was given to patients aged 65 years or older. This study includes the first inpatient encounter between June 2013 and December 31, 2014, for patients having a completed needs assessment and documentation of health literacy data in the medical record.

 

Data Collection

TCN assessment was developed from published patient risk tools and the ITC framework.^2,6,22 The assessment has 10 domains composed of 49 individual items as follows: (1) caregiver support (caregiver support not sufficient for patient needs), (2) transportation (relies on public or others for transportation and misses medical care because of transportation), (3) health care utilization (no primary care physician, unplanned hospitalization in the last year, emergency department [ED] visit in the last 6 months, or home health services in the last 60 days), (4) high-risk medical comorbidities (malnutrition or body mass index <18.5, renal failure, chronic pain, diabetes, heart failure, COPD, or stroke), (5) medication management provider or caregiver concern (cannot provide medication list, >10 preadmission medications, high-risk medications [eg, insulin, warfarin], poor medication understanding, or adherence issue identified), (6) medical devices (vascular access, urinary catheter, wounds, or home supplemental oxygen), (7) functional status (weakness of extremities, limited extremity range of motion, difficulty with mobility, falls at home, or activities of daily living challenges), (8) mental health comorbidities (over the past month has felt down, depressed, or hopeless or over the past month has felt little interest or pleasure in doing things, high-risk alcohol use, or high-risk substance use), (9) communication (limited English proficiency or at risk for limited health literacy), and (10) financial resources (no health insurance, skips or rations medicines because of cost, misses medical care because of cost, or misses medical care because of job).

The 49 items of the TCN assessment were documented as being present or absent by nurse TCCs at the time patients were enrolled in the transitional care program, based on patient and family interview and chart review, and the items were later extracted for analysis. Patients were determined to have a domain-level need if they reported a need on any individual item within that domain, resulting in a binary score (any need present, absent) for each of the 10 TCN domains.

Health literacy was assessed by using the Brief Health Literacy Screen (BHLS), which is administered routinely by nurses at hospital intake and documented in the medical record, with completion rates of approximately 90%.²³ The BHLS is a 3-question subjective health literacy assessment (scoring range 3-15) that has been validated against longer objective measures²⁴ and shown to predict disease control and mortality.^18,25 To improve the stability of scores (for patients who completed the BHLS more than once because of repeat hospitalizations) and to reduce missing values, we calculated the patient’s mean BHLS score for assessments obtained between January 1, 2013, and December 31, 2014. Patients were then categorized as having inadequate health literacy (BHLS ≤ 9) or adequate health literacy (BHLS > 9).^18,25 Demographic information was extracted from patient records and included age, sex (male/female), marital status (married/without a partner), race (white/nonwhite), and years of education. Income level and primary language were not available for analysis.

Statistical Analysis

Patient characteristics and TCNs were summarized by using the frequency and percentages for categorical variables and the mean and standard deviation (SD) for continuous variables. We compared patient characteristics (age, sex, marital status, race, and education) between health literacy groups (inadequate vs adequate) by using χ² or analysis of variance as appropriate. We assessed Pearson correlations among the 10 TCN domains, and we examined differences in reported needs for each of 10 TCN domains by the level of health literacy by using the χ² test. Because the TCN domain of communication included low health literacy as one of its items, we excluded this domain from subsequent analyses. We then compared differences in the number of TCNs documented (scoring range 0-9) by using an independent samples Student t test.

Multivariate logistic regression models were then constructed to examine the independent association of inadequate health literacy with 8 TCN domains while controlling for age, sex, marital status, race, and education. Patients with incomplete demographic data were excluded from these models. Additionally, these analyses excluded 2 TCN domains: the communication domain for reasons noted above and the high-risk medical comorbidity domain because it ended up being positive in 98.4% of patients. Statistical significance was set at an alpha of 0.05. All analyses were performed by using SPSS Statistics for Mac, version 23.0 (IBM Corp., Armonk, New York)

RESULTS

A total of 403 unique patients received the needs assessment, and 384 (95.3%) patients had health literacy data available (Table 1). The number of patients with missing or unknown values were 3 for marital status, 8 for race, and 6 for education. The patients had an average age of 66.9 years (SD = 13.0 years). Among the sample, 209 (54%) were female, 172 (45%) were married, and 291 (75.8%) were white. The average years of education was 12.6 (SD = 2.9 years), and 113 (29%) had inadequate health literacy. Patients with inadequate health literacy completed fewer years of schooling (11.2 vs 13.2; P < 0.001) and were less likely to be married (37% vs 49%; P = 0.031). There was no significant difference in age, sex, or race by level of health literacy.

 

Patients overall had a mean of 4.6 (SD = 1.8) TCN domains with any need reported. The most common domains were high-risk comorbidity (98%), medication management (76%), and healthcare utilization (76%; Table 2). For most domains, the presence of needs was significantly correlated with the presence of needs in multiple other domains (Table 3). Patients with inadequate health literacy had needs in a greater number of TCN domains (mean = 5.29 vs 4.36; P < 0.001).

In unadjusted analysis, patients with inadequate health literacy were significantly more likely to have TCNs in 7 out of the 10 domains (Table 2). These concerns related to caregiver support, transportation, healthcare utilization, presence of a medical device, functional status, mental health comorbidities, and communication. The inadequate and adequate health literacy groups were similar in needs with respect to high-risk comorbidity and finance and borderline nonsignificant for medication management.

In multivariate analyses, 371 patients had complete demographic data and were thus included. After adjustment for age, sex, marital status, race, and education, inadequate health literacy remained significantly associated with reported needs in 2 transitional care domains: inadequate caregiver support (odds ratio [OR], 2.61; 95% confidence interval [CI], 1.37-5.00) and transportation barriers (OR, 1.69; 95% CI, 1.04-2.76; Figure). Other domains approached statistical significance: medical devices (OR, 1.56; 95% CI, 0.96-2.54), functional status (OR, 1.67; 95% CI, 1.00-2.74), and mental health comorbidities (OR, 1.60; 95% CI, 0.98-2.62).

Older age was independently associated with more needs related to medical devices (OR, 1.02; 95% CI, 1.00-1.04), functional status (OR, 1.03; 95% CI, 1.02-1.05), and fewer financial needs (OR, 0.93; 95% CI, 0.91-0.96). Being married or living with a partner was associated with fewer needs related to caregiver support (OR, 0.37; 95% CI, 0.19-0.75) and more device-related needs (OR, 1.60; 95% CI, 1.03-2.49). A higher level of education was associated with fewer transportation needs (OR, 0.89; 95% CI, 0.82-0.97).

DISCUSSION

A structured patient risk factor assessment derived from literature was used to record TCNs in preparation for hospital discharge. On average, patients had needs in about half of the TCN domains (4.6 of 9). The most common areas identified were related to the presence of high-risk comorbidities (98.4%), frequent or prior healthcare utilization (76.6%), medication management (76.3%), functional status (54.9%), and transportation (48.7%). Many of the TCNs were significantly correlated with one another. The prevalence of these needs highlights the importance of using a structured assessment to identify patient concerns so that they may be addressed through discharge planning and follow-up. In addition, using a standardized TCN instrument based on a framework for ITC promotes further research in understanding patient needs and in developing personalized interventions to address them.

As hypothesized, we found that TCNs were more common in patients with inadequate health literacy. After adjustment for demographic factors, inadequate health literacy was significantly associated with transportation barriers and inadequate caregiver support. Analyses also suggested a relationship with needs related to medical devices, functional status, and mental health comorbidities. A review of the literature substantiates a link between inadequate health literacy and these needs and also suggests solutions to address these barriers.

The association with inadequate caregiver support is concerning because there is often a high degree of reliance on caregivers at transitions in care.^3-5 Caregivers are routinely called upon to provide assistance with activities that may be difficult for patients with low health literacy, including medication adherence, provider communication, and self-care activities.^26,27 Our finding that patients with inadequate health literacy are more likely to have inadequate caregiver support indicates additional vulnerability. This may be because of the absence of a caregiver, or in many cases, the presence of a caregiver who is underprepared to assist with care. Prior research has shown that when caregivers are present, up to 33% have low health literacy, even when they are paid nonfamilial caregivers.^26,28 Other studies have noted the inadequacy of information and patient training for caregivers.^29,30 Transitional care programs to improve caregiver understanding have been developed³¹ and have been demonstrated to lower rehospitalization and ED visits.³²

Patients with inadequate health literacy were also more likely to have transportation barriers. Lack of transportation has been recorded as a factor in early hospital readmission in patients with chronic disease,³³ and it has been shown to have a negative effect on a variety of health outcomes.³⁴ A likely link between readmission and lack of transportation is poor follow-up care. Wheeler et al.³⁵ found that 59% of patients expected difficulty keeping postdischarge appointments because of transportation needs. Instead of expecting patients to navigate their own transportation, the Agency for Healthcare Research and Quality recommends identifying community resources for patients with low health literacy.³⁶

In this sample, inadequate health literacy also had near significant associations with TCNs in the use of medical devices, lower functional status, and mental health comorbidities. The use of a medical device, such as home oxygen, is a risk factor for readmission,³⁷ and early reports suggest that interventions, including education related to home oxygen use, can dramatically reduce these readmissions.³⁸ Lower functional capacity and faster functional decline are associated with inadequate health literacy,³⁹ which may have to do with the inability to appropriately utilize health resources.⁴⁰ If so, structured discharge planning could alleviate the known connection between functional impairment and hospital readmissions.⁴¹ A relationship between low health literacy and depression has been demonstrated repeatedly,⁴² with worsened symptoms in those with addiction.⁴³ As has been shown in other domains where health literacy is a factor, literacy-focused interventions provide greater benefits to these depressed patients.⁴⁴

The TCN assessment worked well overall, but certain domains proved less valuable and could be removed in the future. First, it was not useful to separately identify communication barriers, because doing so did not add to information beyond the measurement of health literacy. Second, high-risk comorbidities were ubiquitous within the sample and therefore unhelpful for group comparisons. In hindsight, this is unsurprising because the sample was comprised primarily of elderly patients admitted to medical services. Still, in a younger population or a surgical setting, identifying patients with high-risk medical comorbidities may be more useful.

We acknowledge several limitations of this study. First, the study was performed at a single center, and the TCN assessments were conducted by a small number of registered nurses who received training. Therefore, the results may not generalize to the profile of patient needs at other settings, and the instrument may perform differently when scaled across an organization. Second, the needs assessment was developed for this QI initiative and did not undergo formal validation, although it was developed from published frameworks and similar assessments. Third, for the measure of health literacy, we relied on data collected by nurses as part of their normal workflow. As is often the case with data collected during routine care, the scores are imperfect,⁴⁵ but they have proven to be a valuable and valid indicator of health literacy in our previous research.^18,24,25,46 Fourth, we chose to declare a domain as positive if any item in that domain was positive and to perform a domain-level analysis (for greater clarity). We did not take into account the variable number of items within each domain or attempt to grade their severity, as this would be a subjective exercise and impractical in the discharge planning process. Finally, we were unable to address associations among socioeconomic status,⁴⁷ primary language,⁴⁸ and health literacy, because relevant data were not available for this analysis.

 

CONCLUSION

In this sample of hospitalized patients who were administered a structured needs assessment, patients commonly had needs that placed them at a higher risk of adverse outcomes, such as hospital readmission. Patients with low health literacy had more TCNs that extended beyond the areas that we normally associate with low health literacy, namely patient education and self-management. Healthcare professionals should be aware of the greater likelihood of transportation barriers and inadequate caregiver support among patients with low health literacy. Screening for health literacy and TCN at admission or as part of the discharge planning process will elevate such risks, better positioning clinicians and hospitals to address them as a part of the efforts to ensure a quality transition of care.

Disclosure 

This work was funded by the Centers for Medicare and Medicaid Services (1C1CMS330979) and in part by the National Center for Advancing Translational Sciences (2 UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the funding agencies, which did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication.

Dr. Dittus reports personal fees as a board member of the Robert Wood Johnson Foundation Medical Faculty Scholars Program National Advisory Committee; consultancy fees from the University of Virginia, Indiana University, University of Michigan, Northwestern University, Montana State University, and Purdue University; has grants/grants pending from NIH (research grants), PCORI (research grant), CME (innovation award), VA (training grant); payment for lectures including service on speakers bureaus from Corporate Parity (conference organizer) for the Global Hospital Management & Innovation Summit; and other from Medical Decision Making, Inc. (passive owner); all outside the submitted work. Dr. Kripalani has grants from NIH (research grant), PCORI (research grant), and CMS (QI grant); outside the submitted work. All other authors have nothing to disclose.

A special concern since the institution of hospital readmission penalties¹ is the transitions in care of a patient from one care setting to another, often at hospital discharge. Burke et al.² proposed a framework for an ideal transition in care (ITC) to study and improve transitions from the hospital to home. The features in the ITC were identified based upon their inclusion in the interventions that improved discharge outcomes.^3-5 Inspired by the ITC and other patient risk tools,⁶ we identified 10 domains of transitional care needs ([TCN] specified below), which we define as patient-centered risk factors that should be addressed to foster a safe and effective transition in care.⁷

One particularly important risk factor in patient self-management at transition points is health literacy, a patient’s ability to obtain, understand, and use basic health information and services. Low health literacy affects approximately 26% to 36% of adults in the United States.^8,9 Health literacy is associated with many factors that may affect successful navigation of care transitions, including doctor-patient communication,^10,11 understanding of the medication regimen,¹² and self-management.^13-15 Research has also demonstrated an association between low health literacy and poor outcomes after hospital discharge, including medication errors,¹⁶ 30-day hospital readmission,¹⁷ and mortality.¹⁸ Transitional care initiatives have begun to incorporate health literacy into patient risk assessments⁶ and provide specific attention to low health literacy in interventions to reduce adverse drug events and readmission.^4,19 Training programs for medical students and nurses advise teaching skills in health literacy as part of fostering effective transitions in care.^20,21

Although low health literacy is generally recognized as a barrier to patient education and self-management, little is known about whether patients with low health literacy are more likely to have other risk factors that could further increase their risk for poor transitions in care. A better understanding of associated risks would inform and improve patient care. We hypothesized that TCNs are more common among patients with low health literacy, as compared with those with adequate health literacy. We also aimed to describe the relationship between low health literacy and specific TCNs in order to guide clinical care and future interventions.

METHODS

Setting

The present study is a cross-sectional analysis of data from a quality improvement (QI) intervention that was performed at Vanderbilt University Medical Center, a tertiary care facility in Nashville, Tennessee. The QI intervention, My Health Team (MHT), was funded by the Centers for Medicare and Medicaid Services Innovation Award program. The overall MHT program included outpatient care coordination for chronic disease management as well as a transitional care program that was designed to reduce hospital readmission. The latter included an inpatient needs assessment (which provided data for the present analysis), inpatient intervention, and postdischarge phone follow-up. The MHT initiative was reviewed by the institutional review board (IRB), which deemed it a QI program and granted a waiver of informed consent. The present secondary data analysis was reviewed and approved by the IRB.

Sample

Patients were identified for inclusion in the MHT transitions of care program if the presenting problem for hospital admission was pneumonia, chronic obstructive pulmonary disease (COPD) exacerbation, or decompensated heart failure, as determined by the review of clinical documentation by nurse transition care coordinators (TCCs). Adults over the age of 18 years were eligible, though priority was given to patients aged 65 years or older. This study includes the first inpatient encounter between June 2013 and December 31, 2014, for patients having a completed needs assessment and documentation of health literacy data in the medical record.

 

Data Collection

TCN assessment was developed from published patient risk tools and the ITC framework.^2,6,22 The assessment has 10 domains composed of 49 individual items as follows: (1) caregiver support (caregiver support not sufficient for patient needs), (2) transportation (relies on public or others for transportation and misses medical care because of transportation), (3) health care utilization (no primary care physician, unplanned hospitalization in the last year, emergency department [ED] visit in the last 6 months, or home health services in the last 60 days), (4) high-risk medical comorbidities (malnutrition or body mass index <18.5, renal failure, chronic pain, diabetes, heart failure, COPD, or stroke), (5) medication management provider or caregiver concern (cannot provide medication list, >10 preadmission medications, high-risk medications [eg, insulin, warfarin], poor medication understanding, or adherence issue identified), (6) medical devices (vascular access, urinary catheter, wounds, or home supplemental oxygen), (7) functional status (weakness of extremities, limited extremity range of motion, difficulty with mobility, falls at home, or activities of daily living challenges), (8) mental health comorbidities (over the past month has felt down, depressed, or hopeless or over the past month has felt little interest or pleasure in doing things, high-risk alcohol use, or high-risk substance use), (9) communication (limited English proficiency or at risk for limited health literacy), and (10) financial resources (no health insurance, skips or rations medicines because of cost, misses medical care because of cost, or misses medical care because of job).

The 49 items of the TCN assessment were documented as being present or absent by nurse TCCs at the time patients were enrolled in the transitional care program, based on patient and family interview and chart review, and the items were later extracted for analysis. Patients were determined to have a domain-level need if they reported a need on any individual item within that domain, resulting in a binary score (any need present, absent) for each of the 10 TCN domains.

Health literacy was assessed by using the Brief Health Literacy Screen (BHLS), which is administered routinely by nurses at hospital intake and documented in the medical record, with completion rates of approximately 90%.²³ The BHLS is a 3-question subjective health literacy assessment (scoring range 3-15) that has been validated against longer objective measures²⁴ and shown to predict disease control and mortality.^18,25 To improve the stability of scores (for patients who completed the BHLS more than once because of repeat hospitalizations) and to reduce missing values, we calculated the patient’s mean BHLS score for assessments obtained between January 1, 2013, and December 31, 2014. Patients were then categorized as having inadequate health literacy (BHLS ≤ 9) or adequate health literacy (BHLS > 9).^18,25 Demographic information was extracted from patient records and included age, sex (male/female), marital status (married/without a partner), race (white/nonwhite), and years of education. Income level and primary language were not available for analysis.

Statistical Analysis

Patient characteristics and TCNs were summarized by using the frequency and percentages for categorical variables and the mean and standard deviation (SD) for continuous variables. We compared patient characteristics (age, sex, marital status, race, and education) between health literacy groups (inadequate vs adequate) by using χ² or analysis of variance as appropriate. We assessed Pearson correlations among the 10 TCN domains, and we examined differences in reported needs for each of 10 TCN domains by the level of health literacy by using the χ² test. Because the TCN domain of communication included low health literacy as one of its items, we excluded this domain from subsequent analyses. We then compared differences in the number of TCNs documented (scoring range 0-9) by using an independent samples Student t test.

Multivariate logistic regression models were then constructed to examine the independent association of inadequate health literacy with 8 TCN domains while controlling for age, sex, marital status, race, and education. Patients with incomplete demographic data were excluded from these models. Additionally, these analyses excluded 2 TCN domains: the communication domain for reasons noted above and the high-risk medical comorbidity domain because it ended up being positive in 98.4% of patients. Statistical significance was set at an alpha of 0.05. All analyses were performed by using SPSS Statistics for Mac, version 23.0 (IBM Corp., Armonk, New York)

RESULTS

A total of 403 unique patients received the needs assessment, and 384 (95.3%) patients had health literacy data available (Table 1). The number of patients with missing or unknown values were 3 for marital status, 8 for race, and 6 for education. The patients had an average age of 66.9 years (SD = 13.0 years). Among the sample, 209 (54%) were female, 172 (45%) were married, and 291 (75.8%) were white. The average years of education was 12.6 (SD = 2.9 years), and 113 (29%) had inadequate health literacy. Patients with inadequate health literacy completed fewer years of schooling (11.2 vs 13.2; P < 0.001) and were less likely to be married (37% vs 49%; P = 0.031). There was no significant difference in age, sex, or race by level of health literacy.

 

Patients overall had a mean of 4.6 (SD = 1.8) TCN domains with any need reported. The most common domains were high-risk comorbidity (98%), medication management (76%), and healthcare utilization (76%; Table 2). For most domains, the presence of needs was significantly correlated with the presence of needs in multiple other domains (Table 3). Patients with inadequate health literacy had needs in a greater number of TCN domains (mean = 5.29 vs 4.36; P < 0.001).

In unadjusted analysis, patients with inadequate health literacy were significantly more likely to have TCNs in 7 out of the 10 domains (Table 2). These concerns related to caregiver support, transportation, healthcare utilization, presence of a medical device, functional status, mental health comorbidities, and communication. The inadequate and adequate health literacy groups were similar in needs with respect to high-risk comorbidity and finance and borderline nonsignificant for medication management.

In multivariate analyses, 371 patients had complete demographic data and were thus included. After adjustment for age, sex, marital status, race, and education, inadequate health literacy remained significantly associated with reported needs in 2 transitional care domains: inadequate caregiver support (odds ratio [OR], 2.61; 95% confidence interval [CI], 1.37-5.00) and transportation barriers (OR, 1.69; 95% CI, 1.04-2.76; Figure). Other domains approached statistical significance: medical devices (OR, 1.56; 95% CI, 0.96-2.54), functional status (OR, 1.67; 95% CI, 1.00-2.74), and mental health comorbidities (OR, 1.60; 95% CI, 0.98-2.62).

Older age was independently associated with more needs related to medical devices (OR, 1.02; 95% CI, 1.00-1.04), functional status (OR, 1.03; 95% CI, 1.02-1.05), and fewer financial needs (OR, 0.93; 95% CI, 0.91-0.96). Being married or living with a partner was associated with fewer needs related to caregiver support (OR, 0.37; 95% CI, 0.19-0.75) and more device-related needs (OR, 1.60; 95% CI, 1.03-2.49). A higher level of education was associated with fewer transportation needs (OR, 0.89; 95% CI, 0.82-0.97).

DISCUSSION

A structured patient risk factor assessment derived from literature was used to record TCNs in preparation for hospital discharge. On average, patients had needs in about half of the TCN domains (4.6 of 9). The most common areas identified were related to the presence of high-risk comorbidities (98.4%), frequent or prior healthcare utilization (76.6%), medication management (76.3%), functional status (54.9%), and transportation (48.7%). Many of the TCNs were significantly correlated with one another. The prevalence of these needs highlights the importance of using a structured assessment to identify patient concerns so that they may be addressed through discharge planning and follow-up. In addition, using a standardized TCN instrument based on a framework for ITC promotes further research in understanding patient needs and in developing personalized interventions to address them.

As hypothesized, we found that TCNs were more common in patients with inadequate health literacy. After adjustment for demographic factors, inadequate health literacy was significantly associated with transportation barriers and inadequate caregiver support. Analyses also suggested a relationship with needs related to medical devices, functional status, and mental health comorbidities. A review of the literature substantiates a link between inadequate health literacy and these needs and also suggests solutions to address these barriers.

The association with inadequate caregiver support is concerning because there is often a high degree of reliance on caregivers at transitions in care.^3-5 Caregivers are routinely called upon to provide assistance with activities that may be difficult for patients with low health literacy, including medication adherence, provider communication, and self-care activities.^26,27 Our finding that patients with inadequate health literacy are more likely to have inadequate caregiver support indicates additional vulnerability. This may be because of the absence of a caregiver, or in many cases, the presence of a caregiver who is underprepared to assist with care. Prior research has shown that when caregivers are present, up to 33% have low health literacy, even when they are paid nonfamilial caregivers.^26,28 Other studies have noted the inadequacy of information and patient training for caregivers.^29,30 Transitional care programs to improve caregiver understanding have been developed³¹ and have been demonstrated to lower rehospitalization and ED visits.³²

Patients with inadequate health literacy were also more likely to have transportation barriers. Lack of transportation has been recorded as a factor in early hospital readmission in patients with chronic disease,³³ and it has been shown to have a negative effect on a variety of health outcomes.³⁴ A likely link between readmission and lack of transportation is poor follow-up care. Wheeler et al.³⁵ found that 59% of patients expected difficulty keeping postdischarge appointments because of transportation needs. Instead of expecting patients to navigate their own transportation, the Agency for Healthcare Research and Quality recommends identifying community resources for patients with low health literacy.³⁶

In this sample, inadequate health literacy also had near significant associations with TCNs in the use of medical devices, lower functional status, and mental health comorbidities. The use of a medical device, such as home oxygen, is a risk factor for readmission,³⁷ and early reports suggest that interventions, including education related to home oxygen use, can dramatically reduce these readmissions.³⁸ Lower functional capacity and faster functional decline are associated with inadequate health literacy,³⁹ which may have to do with the inability to appropriately utilize health resources.⁴⁰ If so, structured discharge planning could alleviate the known connection between functional impairment and hospital readmissions.⁴¹ A relationship between low health literacy and depression has been demonstrated repeatedly,⁴² with worsened symptoms in those with addiction.⁴³ As has been shown in other domains where health literacy is a factor, literacy-focused interventions provide greater benefits to these depressed patients.⁴⁴

The TCN assessment worked well overall, but certain domains proved less valuable and could be removed in the future. First, it was not useful to separately identify communication barriers, because doing so did not add to information beyond the measurement of health literacy. Second, high-risk comorbidities were ubiquitous within the sample and therefore unhelpful for group comparisons. In hindsight, this is unsurprising because the sample was comprised primarily of elderly patients admitted to medical services. Still, in a younger population or a surgical setting, identifying patients with high-risk medical comorbidities may be more useful.

We acknowledge several limitations of this study. First, the study was performed at a single center, and the TCN assessments were conducted by a small number of registered nurses who received training. Therefore, the results may not generalize to the profile of patient needs at other settings, and the instrument may perform differently when scaled across an organization. Second, the needs assessment was developed for this QI initiative and did not undergo formal validation, although it was developed from published frameworks and similar assessments. Third, for the measure of health literacy, we relied on data collected by nurses as part of their normal workflow. As is often the case with data collected during routine care, the scores are imperfect,⁴⁵ but they have proven to be a valuable and valid indicator of health literacy in our previous research.^18,24,25,46 Fourth, we chose to declare a domain as positive if any item in that domain was positive and to perform a domain-level analysis (for greater clarity). We did not take into account the variable number of items within each domain or attempt to grade their severity, as this would be a subjective exercise and impractical in the discharge planning process. Finally, we were unable to address associations among socioeconomic status,⁴⁷ primary language,⁴⁸ and health literacy, because relevant data were not available for this analysis.

 

CONCLUSION

In this sample of hospitalized patients who were administered a structured needs assessment, patients commonly had needs that placed them at a higher risk of adverse outcomes, such as hospital readmission. Patients with low health literacy had more TCNs that extended beyond the areas that we normally associate with low health literacy, namely patient education and self-management. Healthcare professionals should be aware of the greater likelihood of transportation barriers and inadequate caregiver support among patients with low health literacy. Screening for health literacy and TCN at admission or as part of the discharge planning process will elevate such risks, better positioning clinicians and hospitals to address them as a part of the efforts to ensure a quality transition of care.

Disclosure 

This work was funded by the Centers for Medicare and Medicaid Services (1C1CMS330979) and in part by the National Center for Advancing Translational Sciences (2 UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the funding agencies, which did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication.

Dr. Dittus reports personal fees as a board member of the Robert Wood Johnson Foundation Medical Faculty Scholars Program National Advisory Committee; consultancy fees from the University of Virginia, Indiana University, University of Michigan, Northwestern University, Montana State University, and Purdue University; has grants/grants pending from NIH (research grants), PCORI (research grant), CME (innovation award), VA (training grant); payment for lectures including service on speakers bureaus from Corporate Parity (conference organizer) for the Global Hospital Management & Innovation Summit; and other from Medical Decision Making, Inc. (passive owner); all outside the submitted work. Dr. Kripalani has grants from NIH (research grant), PCORI (research grant), and CMS (QI grant); outside the submitted work. All other authors have nothing to disclose.

A special concern since the institution of hospital readmission penalties¹ is the transitions in care of a patient from one care setting to another, often at hospital discharge. Burke et al.² proposed a framework for an ideal transition in care (ITC) to study and improve transitions from the hospital to home. The features in the ITC were identified based upon their inclusion in the interventions that improved discharge outcomes.^3-5 Inspired by the ITC and other patient risk tools,⁶ we identified 10 domains of transitional care needs ([TCN] specified below), which we define as patient-centered risk factors that should be addressed to foster a safe and effective transition in care.⁷

One particularly important risk factor in patient self-management at transition points is health literacy, a patient’s ability to obtain, understand, and use basic health information and services. Low health literacy affects approximately 26% to 36% of adults in the United States.^8,9 Health literacy is associated with many factors that may affect successful navigation of care transitions, including doctor-patient communication,^10,11 understanding of the medication regimen,¹² and self-management.^13-15 Research has also demonstrated an association between low health literacy and poor outcomes after hospital discharge, including medication errors,¹⁶ 30-day hospital readmission,¹⁷ and mortality.¹⁸ Transitional care initiatives have begun to incorporate health literacy into patient risk assessments⁶ and provide specific attention to low health literacy in interventions to reduce adverse drug events and readmission.^4,19 Training programs for medical students and nurses advise teaching skills in health literacy as part of fostering effective transitions in care.^20,21

Although low health literacy is generally recognized as a barrier to patient education and self-management, little is known about whether patients with low health literacy are more likely to have other risk factors that could further increase their risk for poor transitions in care. A better understanding of associated risks would inform and improve patient care. We hypothesized that TCNs are more common among patients with low health literacy, as compared with those with adequate health literacy. We also aimed to describe the relationship between low health literacy and specific TCNs in order to guide clinical care and future interventions.

METHODS

Setting

The present study is a cross-sectional analysis of data from a quality improvement (QI) intervention that was performed at Vanderbilt University Medical Center, a tertiary care facility in Nashville, Tennessee. The QI intervention, My Health Team (MHT), was funded by the Centers for Medicare and Medicaid Services Innovation Award program. The overall MHT program included outpatient care coordination for chronic disease management as well as a transitional care program that was designed to reduce hospital readmission. The latter included an inpatient needs assessment (which provided data for the present analysis), inpatient intervention, and postdischarge phone follow-up. The MHT initiative was reviewed by the institutional review board (IRB), which deemed it a QI program and granted a waiver of informed consent. The present secondary data analysis was reviewed and approved by the IRB.

Sample

Patients were identified for inclusion in the MHT transitions of care program if the presenting problem for hospital admission was pneumonia, chronic obstructive pulmonary disease (COPD) exacerbation, or decompensated heart failure, as determined by the review of clinical documentation by nurse transition care coordinators (TCCs). Adults over the age of 18 years were eligible, though priority was given to patients aged 65 years or older. This study includes the first inpatient encounter between June 2013 and December 31, 2014, for patients having a completed needs assessment and documentation of health literacy data in the medical record.

 

Data Collection

TCN assessment was developed from published patient risk tools and the ITC framework.^2,6,22 The assessment has 10 domains composed of 49 individual items as follows: (1) caregiver support (caregiver support not sufficient for patient needs), (2) transportation (relies on public or others for transportation and misses medical care because of transportation), (3) health care utilization (no primary care physician, unplanned hospitalization in the last year, emergency department [ED] visit in the last 6 months, or home health services in the last 60 days), (4) high-risk medical comorbidities (malnutrition or body mass index <18.5, renal failure, chronic pain, diabetes, heart failure, COPD, or stroke), (5) medication management provider or caregiver concern (cannot provide medication list, >10 preadmission medications, high-risk medications [eg, insulin, warfarin], poor medication understanding, or adherence issue identified), (6) medical devices (vascular access, urinary catheter, wounds, or home supplemental oxygen), (7) functional status (weakness of extremities, limited extremity range of motion, difficulty with mobility, falls at home, or activities of daily living challenges), (8) mental health comorbidities (over the past month has felt down, depressed, or hopeless or over the past month has felt little interest or pleasure in doing things, high-risk alcohol use, or high-risk substance use), (9) communication (limited English proficiency or at risk for limited health literacy), and (10) financial resources (no health insurance, skips or rations medicines because of cost, misses medical care because of cost, or misses medical care because of job).

The 49 items of the TCN assessment were documented as being present or absent by nurse TCCs at the time patients were enrolled in the transitional care program, based on patient and family interview and chart review, and the items were later extracted for analysis. Patients were determined to have a domain-level need if they reported a need on any individual item within that domain, resulting in a binary score (any need present, absent) for each of the 10 TCN domains.

Health literacy was assessed by using the Brief Health Literacy Screen (BHLS), which is administered routinely by nurses at hospital intake and documented in the medical record, with completion rates of approximately 90%.²³ The BHLS is a 3-question subjective health literacy assessment (scoring range 3-15) that has been validated against longer objective measures²⁴ and shown to predict disease control and mortality.^18,25 To improve the stability of scores (for patients who completed the BHLS more than once because of repeat hospitalizations) and to reduce missing values, we calculated the patient’s mean BHLS score for assessments obtained between January 1, 2013, and December 31, 2014. Patients were then categorized as having inadequate health literacy (BHLS ≤ 9) or adequate health literacy (BHLS > 9).^18,25 Demographic information was extracted from patient records and included age, sex (male/female), marital status (married/without a partner), race (white/nonwhite), and years of education. Income level and primary language were not available for analysis.

Statistical Analysis

Patient characteristics and TCNs were summarized by using the frequency and percentages for categorical variables and the mean and standard deviation (SD) for continuous variables. We compared patient characteristics (age, sex, marital status, race, and education) between health literacy groups (inadequate vs adequate) by using χ² or analysis of variance as appropriate. We assessed Pearson correlations among the 10 TCN domains, and we examined differences in reported needs for each of 10 TCN domains by the level of health literacy by using the χ² test. Because the TCN domain of communication included low health literacy as one of its items, we excluded this domain from subsequent analyses. We then compared differences in the number of TCNs documented (scoring range 0-9) by using an independent samples Student t test.

Multivariate logistic regression models were then constructed to examine the independent association of inadequate health literacy with 8 TCN domains while controlling for age, sex, marital status, race, and education. Patients with incomplete demographic data were excluded from these models. Additionally, these analyses excluded 2 TCN domains: the communication domain for reasons noted above and the high-risk medical comorbidity domain because it ended up being positive in 98.4% of patients. Statistical significance was set at an alpha of 0.05. All analyses were performed by using SPSS Statistics for Mac, version 23.0 (IBM Corp., Armonk, New York)

RESULTS

A total of 403 unique patients received the needs assessment, and 384 (95.3%) patients had health literacy data available (Table 1). The number of patients with missing or unknown values were 3 for marital status, 8 for race, and 6 for education. The patients had an average age of 66.9 years (SD = 13.0 years). Among the sample, 209 (54%) were female, 172 (45%) were married, and 291 (75.8%) were white. The average years of education was 12.6 (SD = 2.9 years), and 113 (29%) had inadequate health literacy. Patients with inadequate health literacy completed fewer years of schooling (11.2 vs 13.2; P < 0.001) and were less likely to be married (37% vs 49%; P = 0.031). There was no significant difference in age, sex, or race by level of health literacy.

 

Patients overall had a mean of 4.6 (SD = 1.8) TCN domains with any need reported. The most common domains were high-risk comorbidity (98%), medication management (76%), and healthcare utilization (76%; Table 2). For most domains, the presence of needs was significantly correlated with the presence of needs in multiple other domains (Table 3). Patients with inadequate health literacy had needs in a greater number of TCN domains (mean = 5.29 vs 4.36; P < 0.001).

In unadjusted analysis, patients with inadequate health literacy were significantly more likely to have TCNs in 7 out of the 10 domains (Table 2). These concerns related to caregiver support, transportation, healthcare utilization, presence of a medical device, functional status, mental health comorbidities, and communication. The inadequate and adequate health literacy groups were similar in needs with respect to high-risk comorbidity and finance and borderline nonsignificant for medication management.

In multivariate analyses, 371 patients had complete demographic data and were thus included. After adjustment for age, sex, marital status, race, and education, inadequate health literacy remained significantly associated with reported needs in 2 transitional care domains: inadequate caregiver support (odds ratio [OR], 2.61; 95% confidence interval [CI], 1.37-5.00) and transportation barriers (OR, 1.69; 95% CI, 1.04-2.76; Figure). Other domains approached statistical significance: medical devices (OR, 1.56; 95% CI, 0.96-2.54), functional status (OR, 1.67; 95% CI, 1.00-2.74), and mental health comorbidities (OR, 1.60; 95% CI, 0.98-2.62).

Older age was independently associated with more needs related to medical devices (OR, 1.02; 95% CI, 1.00-1.04), functional status (OR, 1.03; 95% CI, 1.02-1.05), and fewer financial needs (OR, 0.93; 95% CI, 0.91-0.96). Being married or living with a partner was associated with fewer needs related to caregiver support (OR, 0.37; 95% CI, 0.19-0.75) and more device-related needs (OR, 1.60; 95% CI, 1.03-2.49). A higher level of education was associated with fewer transportation needs (OR, 0.89; 95% CI, 0.82-0.97).

DISCUSSION

A structured patient risk factor assessment derived from literature was used to record TCNs in preparation for hospital discharge. On average, patients had needs in about half of the TCN domains (4.6 of 9). The most common areas identified were related to the presence of high-risk comorbidities (98.4%), frequent or prior healthcare utilization (76.6%), medication management (76.3%), functional status (54.9%), and transportation (48.7%). Many of the TCNs were significantly correlated with one another. The prevalence of these needs highlights the importance of using a structured assessment to identify patient concerns so that they may be addressed through discharge planning and follow-up. In addition, using a standardized TCN instrument based on a framework for ITC promotes further research in understanding patient needs and in developing personalized interventions to address them.

As hypothesized, we found that TCNs were more common in patients with inadequate health literacy. After adjustment for demographic factors, inadequate health literacy was significantly associated with transportation barriers and inadequate caregiver support. Analyses also suggested a relationship with needs related to medical devices, functional status, and mental health comorbidities. A review of the literature substantiates a link between inadequate health literacy and these needs and also suggests solutions to address these barriers.

The association with inadequate caregiver support is concerning because there is often a high degree of reliance on caregivers at transitions in care.^3-5 Caregivers are routinely called upon to provide assistance with activities that may be difficult for patients with low health literacy, including medication adherence, provider communication, and self-care activities.^26,27 Our finding that patients with inadequate health literacy are more likely to have inadequate caregiver support indicates additional vulnerability. This may be because of the absence of a caregiver, or in many cases, the presence of a caregiver who is underprepared to assist with care. Prior research has shown that when caregivers are present, up to 33% have low health literacy, even when they are paid nonfamilial caregivers.^26,28 Other studies have noted the inadequacy of information and patient training for caregivers.^29,30 Transitional care programs to improve caregiver understanding have been developed³¹ and have been demonstrated to lower rehospitalization and ED visits.³²

Patients with inadequate health literacy were also more likely to have transportation barriers. Lack of transportation has been recorded as a factor in early hospital readmission in patients with chronic disease,³³ and it has been shown to have a negative effect on a variety of health outcomes.³⁴ A likely link between readmission and lack of transportation is poor follow-up care. Wheeler et al.³⁵ found that 59% of patients expected difficulty keeping postdischarge appointments because of transportation needs. Instead of expecting patients to navigate their own transportation, the Agency for Healthcare Research and Quality recommends identifying community resources for patients with low health literacy.³⁶

In this sample, inadequate health literacy also had near significant associations with TCNs in the use of medical devices, lower functional status, and mental health comorbidities. The use of a medical device, such as home oxygen, is a risk factor for readmission,³⁷ and early reports suggest that interventions, including education related to home oxygen use, can dramatically reduce these readmissions.³⁸ Lower functional capacity and faster functional decline are associated with inadequate health literacy,³⁹ which may have to do with the inability to appropriately utilize health resources.⁴⁰ If so, structured discharge planning could alleviate the known connection between functional impairment and hospital readmissions.⁴¹ A relationship between low health literacy and depression has been demonstrated repeatedly,⁴² with worsened symptoms in those with addiction.⁴³ As has been shown in other domains where health literacy is a factor, literacy-focused interventions provide greater benefits to these depressed patients.⁴⁴

The TCN assessment worked well overall, but certain domains proved less valuable and could be removed in the future. First, it was not useful to separately identify communication barriers, because doing so did not add to information beyond the measurement of health literacy. Second, high-risk comorbidities were ubiquitous within the sample and therefore unhelpful for group comparisons. In hindsight, this is unsurprising because the sample was comprised primarily of elderly patients admitted to medical services. Still, in a younger population or a surgical setting, identifying patients with high-risk medical comorbidities may be more useful.

We acknowledge several limitations of this study. First, the study was performed at a single center, and the TCN assessments were conducted by a small number of registered nurses who received training. Therefore, the results may not generalize to the profile of patient needs at other settings, and the instrument may perform differently when scaled across an organization. Second, the needs assessment was developed for this QI initiative and did not undergo formal validation, although it was developed from published frameworks and similar assessments. Third, for the measure of health literacy, we relied on data collected by nurses as part of their normal workflow. As is often the case with data collected during routine care, the scores are imperfect,⁴⁵ but they have proven to be a valuable and valid indicator of health literacy in our previous research.^18,24,25,46 Fourth, we chose to declare a domain as positive if any item in that domain was positive and to perform a domain-level analysis (for greater clarity). We did not take into account the variable number of items within each domain or attempt to grade their severity, as this would be a subjective exercise and impractical in the discharge planning process. Finally, we were unable to address associations among socioeconomic status,⁴⁷ primary language,⁴⁸ and health literacy, because relevant data were not available for this analysis.

 

CONCLUSION

In this sample of hospitalized patients who were administered a structured needs assessment, patients commonly had needs that placed them at a higher risk of adverse outcomes, such as hospital readmission. Patients with low health literacy had more TCNs that extended beyond the areas that we normally associate with low health literacy, namely patient education and self-management. Healthcare professionals should be aware of the greater likelihood of transportation barriers and inadequate caregiver support among patients with low health literacy. Screening for health literacy and TCN at admission or as part of the discharge planning process will elevate such risks, better positioning clinicians and hospitals to address them as a part of the efforts to ensure a quality transition of care.

Disclosure 

This work was funded by the Centers for Medicare and Medicaid Services (1C1CMS330979) and in part by the National Center for Advancing Translational Sciences (2 UL1 TR000445-06). The content is solely the responsibility of the authors and does not necessarily represent official views of the funding agencies, which did not participate in the planning, collection, analysis, or interpretation of data or in the decision to submit for publication.

Dr. Dittus reports personal fees as a board member of the Robert Wood Johnson Foundation Medical Faculty Scholars Program National Advisory Committee; consultancy fees from the University of Virginia, Indiana University, University of Michigan, Northwestern University, Montana State University, and Purdue University; has grants/grants pending from NIH (research grants), PCORI (research grant), CME (innovation award), VA (training grant); payment for lectures including service on speakers bureaus from Corporate Parity (conference organizer) for the Global Hospital Management & Innovation Summit; and other from Medical Decision Making, Inc. (passive owner); all outside the submitted work. Dr. Kripalani has grants from NIH (research grant), PCORI (research grant), and CMS (QI grant); outside the submitted work. All other authors have nothing to disclose.