Mr. Q, age 36, has a history of schizophrenia. He is brought to the hospital due to persistent auditory hallucinations and paranoid delusions. His history documents a trial and failure of risperidone, 4 mg twice daily, and aripiprazole, 20 mg/d. Based on this, the treatment team initiates haloperidol, 5 mg twice daily. Because he experiences persistent auditory hallucinations and paranoid delusions, Mr. Q is titrated to increasing doses of haloperidol over 2 weeks during the course of the hospitalization. Once Mr. Q is receiving a total haloperidol dose of 30 mg/d, the team decides to obtain a serum haloperidol level due to his persistent psychotic symptoms and the development of drug-induced parkinsonism. His serum haloperidol level is 24 ng/mL, which is within the expected range for his dose, but above the therapeutic window for efficacy. The team decides that the severity of Mr. Q’s illness and documented treatment resistance (failing at least 2 adequate trials of antipsychotics) warrant a trial of clozapine.

Despite a long history of therapeutic drug monitoring (TDM) within psychiatry, routine monitoring of antipsychotic serum levels has not been unanimously adopted as standard practice. Clinical practice typically results in mostly a subjective assessment of the safety and efficacy of antipsychotics. This practice is in contrast to agents such as valproic acid and lithium, which are routinely monitored for safety and efficacy using both subjective and laboratory measures. Clinicians may adhere to these monitoring practices for lithium and valproic acid because of these agents’ narrow “therapeutic window” between toxicity and efficacy. However, antipsychotics can be viewed in a similar fashion.

To help conceptualize the therapeutic window for antipsychotics, it is important to understand that in most cases, the pharmacologic target for antipsychotics is dopamine (D2)receptor antagonism between 60% to 80%.¹ Total drug exposure would thus determine a patient’s likelihood of minimizing positive symptoms, or exposure to adverse effects related to total dopamine antagonism. Serum drug concentrations are a better metric than total daily dose for determining drug exposure and achieving the pharmacologic target.² Evaluating serum antipsychotic levels also is a better method of determining true treatment failure than relying on the clinical judgment of the treating psychiatrist.³

Pros and cons of TDM

Benefits of using TDM for patients being treated with antipsychotics include⁴:

• ensuring adherence
• quantitatively adjusting dosages for medication interactions or genetic variations
• ensuring an adequate trial of a medication before considering it a treatment failure.

Potential drawbacks to TDM include:

• Delayed results. Access to expeditious testing may not be possible in certain laboratories, and this may require send-out testing, which could result in a delay in obtaining results. Continued advocacy and research on the value of TDM in antipsychotics may improve access to these resources in the future. Nonetheless, obtaining antipsychotic serum levels will still give clinicians insight into the antipsychotic exposure at a given dose. Further, obtaining antipsychotic serum levels may strengthen decisions about treatment resistance and the assessment of interactions, adherence, or the likelihood of adverse effects.
• Lack of guidance. Unfortunately, there is no established guidance outlining what to do once antipsychotic serum levels are obtained. The correlation of serum levels of commonly used second-generation antipsychotics with clinical efficacy needs to be more closely investigated; however, certain agents do have more data associated with appropriate ranges for efficacy/toxicity. While researchers should continue to study the precise relationship between antipsychotic serum levels and effect, clinicians still have resources available to help determine what the expected serum value for a given patient may be. Knowing an expected serum level may help clinicians determine whether there is an unknown interaction or genetic variation that is causing lower- or higher-than-expected levels. This may also help determine whether a patient is adhering to their medication regimen.

Growing evidence for TDM

In recent years, evidence supporting the use of TDM in patients receiving antipsychotics has been increasing, and recommendations from consensus groups have been strengthened. One of the most comprehensive assessments of these practices was published by the Arbeitsgemeinschaft für Neuropsychopharmakologie und Pharmakopsychiatrie (AGNP), a German-based psychopharmacology group consisting of researchers and clinicians. In 2018, the AGNP published consensus guidelines for TDM in various neuropsychiatric medication classes and recommendations for collecting, interpreting, and decision-making elements of the process.⁵ The Table⁵ lists the AGNP-recommended therapeutic serum range for several commonly used second-generation antipsychotics.

Researchers should be encouraged to contribute to the body of knowledge on the correlation of clinical response to serum level. However, there is compelling evidence for the use of TDM across many first- and second-generation antipsychotics. Of the most common, haloperidol and clozapine have evidence of a therapeutic range that is better correlated with serum level than daily dose. Specifically, haloperidol appears to lose benefit with dose increases beyond serum levels of approximately 10 ng/mL.⁶

Continue to: Clozapine levels may be...

Clozapine levels may be reported by measuring the metabolite norclozapine, which is not correlated with efficacy, or as a total level (combination of clozapine and norclozapine). While norclozapine is not associated with efficacy, the ratio of clozapine to norclozapine may indicate adherence to the medication, or any enzymatic modulation (genetic or drug–drug interaction) that may increase or decrease total exposure. A ratio of 1.5 to 2.0 (clozapine to norclozapine) is optimal; a ratio <0.5 may indicate nonadherence; and a ratio >2.0 may indicate inhibited drug clearance. A 12-hour serum clozapine level of ≥350 ng/mL is more likely to predict treatment response.⁷

CASE CONTINUED

Mr. Q is carefully tapered from haloperidol while initiating clozapine at 25 mg/d. As he is titrated on clozapine, Mr. Q’s serum levels are periodically checked and compared with expected levels and levels associated with efficacy. Eventually, Mr. Q is titrated to a clozapine dose of 400 mg/d at bedtime.

While receiving clozapine for 4 weeks, Mr. Q’s psychotic symptoms resolve, and he is scheduled for follow-up in the outpatient clozapine clinic.

Related Resources

• De Leon J. A critical commentary on the 2017 AGNP consensus guidelines for therapeutic drug monitoring in neuropsychopharmacology. Pharmacopsychiatry. 2018; 51(1-02):63-68.
• Meyer JM. Is monitoring of plasma antipsychotic levels useful? Current Psychiatry. 2015;14(11):16,19-20.

Drug Brand Names

Aripiprazole • Abilify
Clozapine • Clozaril
Fluphenazine • Prolixin
Haloperidol • Haldol
Lithium • Eskalith, Lithobid
Lurasidone • Latuda
Olanzapine • Zyprexa
Perphenazine • Trilafon
Quetiapine • Seroquel
Risperidone • Risperdal
Valproic acid • Depakene, Depakote
Ziprasidone • Geodon

Mr. Q, age 36, has a history of schizophrenia. He is brought to the hospital due to persistent auditory hallucinations and paranoid delusions. His history documents a trial and failure of risperidone, 4 mg twice daily, and aripiprazole, 20 mg/d. Based on this, the treatment team initiates haloperidol, 5 mg twice daily. Because he experiences persistent auditory hallucinations and paranoid delusions, Mr. Q is titrated to increasing doses of haloperidol over 2 weeks during the course of the hospitalization. Once Mr. Q is receiving a total haloperidol dose of 30 mg/d, the team decides to obtain a serum haloperidol level due to his persistent psychotic symptoms and the development of drug-induced parkinsonism. His serum haloperidol level is 24 ng/mL, which is within the expected range for his dose, but above the therapeutic window for efficacy. The team decides that the severity of Mr. Q’s illness and documented treatment resistance (failing at least 2 adequate trials of antipsychotics) warrant a trial of clozapine.

Despite a long history of therapeutic drug monitoring (TDM) within psychiatry, routine monitoring of antipsychotic serum levels has not been unanimously adopted as standard practice. Clinical practice typically results in mostly a subjective assessment of the safety and efficacy of antipsychotics. This practice is in contrast to agents such as valproic acid and lithium, which are routinely monitored for safety and efficacy using both subjective and laboratory measures. Clinicians may adhere to these monitoring practices for lithium and valproic acid because of these agents’ narrow “therapeutic window” between toxicity and efficacy. However, antipsychotics can be viewed in a similar fashion.

To help conceptualize the therapeutic window for antipsychotics, it is important to understand that in most cases, the pharmacologic target for antipsychotics is dopamine (D2)receptor antagonism between 60% to 80%.¹ Total drug exposure would thus determine a patient’s likelihood of minimizing positive symptoms, or exposure to adverse effects related to total dopamine antagonism. Serum drug concentrations are a better metric than total daily dose for determining drug exposure and achieving the pharmacologic target.² Evaluating serum antipsychotic levels also is a better method of determining true treatment failure than relying on the clinical judgment of the treating psychiatrist.³

Pros and cons of TDM

Benefits of using TDM for patients being treated with antipsychotics include⁴:

• ensuring adherence
• quantitatively adjusting dosages for medication interactions or genetic variations
• ensuring an adequate trial of a medication before considering it a treatment failure.

Potential drawbacks to TDM include:

• Delayed results. Access to expeditious testing may not be possible in certain laboratories, and this may require send-out testing, which could result in a delay in obtaining results. Continued advocacy and research on the value of TDM in antipsychotics may improve access to these resources in the future. Nonetheless, obtaining antipsychotic serum levels will still give clinicians insight into the antipsychotic exposure at a given dose. Further, obtaining antipsychotic serum levels may strengthen decisions about treatment resistance and the assessment of interactions, adherence, or the likelihood of adverse effects.
• Lack of guidance. Unfortunately, there is no established guidance outlining what to do once antipsychotic serum levels are obtained. The correlation of serum levels of commonly used second-generation antipsychotics with clinical efficacy needs to be more closely investigated; however, certain agents do have more data associated with appropriate ranges for efficacy/toxicity. While researchers should continue to study the precise relationship between antipsychotic serum levels and effect, clinicians still have resources available to help determine what the expected serum value for a given patient may be. Knowing an expected serum level may help clinicians determine whether there is an unknown interaction or genetic variation that is causing lower- or higher-than-expected levels. This may also help determine whether a patient is adhering to their medication regimen.

Growing evidence for TDM

In recent years, evidence supporting the use of TDM in patients receiving antipsychotics has been increasing, and recommendations from consensus groups have been strengthened. One of the most comprehensive assessments of these practices was published by the Arbeitsgemeinschaft für Neuropsychopharmakologie und Pharmakopsychiatrie (AGNP), a German-based psychopharmacology group consisting of researchers and clinicians. In 2018, the AGNP published consensus guidelines for TDM in various neuropsychiatric medication classes and recommendations for collecting, interpreting, and decision-making elements of the process.⁵ The Table⁵ lists the AGNP-recommended therapeutic serum range for several commonly used second-generation antipsychotics.

Researchers should be encouraged to contribute to the body of knowledge on the correlation of clinical response to serum level. However, there is compelling evidence for the use of TDM across many first- and second-generation antipsychotics. Of the most common, haloperidol and clozapine have evidence of a therapeutic range that is better correlated with serum level than daily dose. Specifically, haloperidol appears to lose benefit with dose increases beyond serum levels of approximately 10 ng/mL.⁶

Continue to: Clozapine levels may be...

Clozapine levels may be reported by measuring the metabolite norclozapine, which is not correlated with efficacy, or as a total level (combination of clozapine and norclozapine). While norclozapine is not associated with efficacy, the ratio of clozapine to norclozapine may indicate adherence to the medication, or any enzymatic modulation (genetic or drug–drug interaction) that may increase or decrease total exposure. A ratio of 1.5 to 2.0 (clozapine to norclozapine) is optimal; a ratio <0.5 may indicate nonadherence; and a ratio >2.0 may indicate inhibited drug clearance. A 12-hour serum clozapine level of ≥350 ng/mL is more likely to predict treatment response.⁷

CASE CONTINUED

Mr. Q is carefully tapered from haloperidol while initiating clozapine at 25 mg/d. As he is titrated on clozapine, Mr. Q’s serum levels are periodically checked and compared with expected levels and levels associated with efficacy. Eventually, Mr. Q is titrated to a clozapine dose of 400 mg/d at bedtime.

While receiving clozapine for 4 weeks, Mr. Q’s psychotic symptoms resolve, and he is scheduled for follow-up in the outpatient clozapine clinic.

Related Resources

• De Leon J. A critical commentary on the 2017 AGNP consensus guidelines for therapeutic drug monitoring in neuropsychopharmacology. Pharmacopsychiatry. 2018; 51(1-02):63-68.
• Meyer JM. Is monitoring of plasma antipsychotic levels useful? Current Psychiatry. 2015;14(11):16,19-20.

Drug Brand Names

Aripiprazole • Abilify
Clozapine • Clozaril
Fluphenazine • Prolixin
Haloperidol • Haldol
Lithium • Eskalith, Lithobid
Lurasidone • Latuda
Olanzapine • Zyprexa
Perphenazine • Trilafon
Quetiapine • Seroquel
Risperidone • Risperdal
Valproic acid • Depakene, Depakote
Ziprasidone • Geodon

Mr. Q, age 36, has a history of schizophrenia. He is brought to the hospital due to persistent auditory hallucinations and paranoid delusions. His history documents a trial and failure of risperidone, 4 mg twice daily, and aripiprazole, 20 mg/d. Based on this, the treatment team initiates haloperidol, 5 mg twice daily. Because he experiences persistent auditory hallucinations and paranoid delusions, Mr. Q is titrated to increasing doses of haloperidol over 2 weeks during the course of the hospitalization. Once Mr. Q is receiving a total haloperidol dose of 30 mg/d, the team decides to obtain a serum haloperidol level due to his persistent psychotic symptoms and the development of drug-induced parkinsonism. His serum haloperidol level is 24 ng/mL, which is within the expected range for his dose, but above the therapeutic window for efficacy. The team decides that the severity of Mr. Q’s illness and documented treatment resistance (failing at least 2 adequate trials of antipsychotics) warrant a trial of clozapine.

Despite a long history of therapeutic drug monitoring (TDM) within psychiatry, routine monitoring of antipsychotic serum levels has not been unanimously adopted as standard practice. Clinical practice typically results in mostly a subjective assessment of the safety and efficacy of antipsychotics. This practice is in contrast to agents such as valproic acid and lithium, which are routinely monitored for safety and efficacy using both subjective and laboratory measures. Clinicians may adhere to these monitoring practices for lithium and valproic acid because of these agents’ narrow “therapeutic window” between toxicity and efficacy. However, antipsychotics can be viewed in a similar fashion.

To help conceptualize the therapeutic window for antipsychotics, it is important to understand that in most cases, the pharmacologic target for antipsychotics is dopamine (D2)receptor antagonism between 60% to 80%.¹ Total drug exposure would thus determine a patient’s likelihood of minimizing positive symptoms, or exposure to adverse effects related to total dopamine antagonism. Serum drug concentrations are a better metric than total daily dose for determining drug exposure and achieving the pharmacologic target.² Evaluating serum antipsychotic levels also is a better method of determining true treatment failure than relying on the clinical judgment of the treating psychiatrist.³

Pros and cons of TDM

Benefits of using TDM for patients being treated with antipsychotics include⁴:

• ensuring adherence
• quantitatively adjusting dosages for medication interactions or genetic variations
• ensuring an adequate trial of a medication before considering it a treatment failure.

Potential drawbacks to TDM include:

• Delayed results. Access to expeditious testing may not be possible in certain laboratories, and this may require send-out testing, which could result in a delay in obtaining results. Continued advocacy and research on the value of TDM in antipsychotics may improve access to these resources in the future. Nonetheless, obtaining antipsychotic serum levels will still give clinicians insight into the antipsychotic exposure at a given dose. Further, obtaining antipsychotic serum levels may strengthen decisions about treatment resistance and the assessment of interactions, adherence, or the likelihood of adverse effects.
• Lack of guidance. Unfortunately, there is no established guidance outlining what to do once antipsychotic serum levels are obtained. The correlation of serum levels of commonly used second-generation antipsychotics with clinical efficacy needs to be more closely investigated; however, certain agents do have more data associated with appropriate ranges for efficacy/toxicity. While researchers should continue to study the precise relationship between antipsychotic serum levels and effect, clinicians still have resources available to help determine what the expected serum value for a given patient may be. Knowing an expected serum level may help clinicians determine whether there is an unknown interaction or genetic variation that is causing lower- or higher-than-expected levels. This may also help determine whether a patient is adhering to their medication regimen.

Growing evidence for TDM

In recent years, evidence supporting the use of TDM in patients receiving antipsychotics has been increasing, and recommendations from consensus groups have been strengthened. One of the most comprehensive assessments of these practices was published by the Arbeitsgemeinschaft für Neuropsychopharmakologie und Pharmakopsychiatrie (AGNP), a German-based psychopharmacology group consisting of researchers and clinicians. In 2018, the AGNP published consensus guidelines for TDM in various neuropsychiatric medication classes and recommendations for collecting, interpreting, and decision-making elements of the process.⁵ The Table⁵ lists the AGNP-recommended therapeutic serum range for several commonly used second-generation antipsychotics.

Researchers should be encouraged to contribute to the body of knowledge on the correlation of clinical response to serum level. However, there is compelling evidence for the use of TDM across many first- and second-generation antipsychotics. Of the most common, haloperidol and clozapine have evidence of a therapeutic range that is better correlated with serum level than daily dose. Specifically, haloperidol appears to lose benefit with dose increases beyond serum levels of approximately 10 ng/mL.⁶

Continue to: Clozapine levels may be...

Clozapine levels may be reported by measuring the metabolite norclozapine, which is not correlated with efficacy, or as a total level (combination of clozapine and norclozapine). While norclozapine is not associated with efficacy, the ratio of clozapine to norclozapine may indicate adherence to the medication, or any enzymatic modulation (genetic or drug–drug interaction) that may increase or decrease total exposure. A ratio of 1.5 to 2.0 (clozapine to norclozapine) is optimal; a ratio <0.5 may indicate nonadherence; and a ratio >2.0 may indicate inhibited drug clearance. A 12-hour serum clozapine level of ≥350 ng/mL is more likely to predict treatment response.⁷

CASE CONTINUED

Mr. Q is carefully tapered from haloperidol while initiating clozapine at 25 mg/d. As he is titrated on clozapine, Mr. Q’s serum levels are periodically checked and compared with expected levels and levels associated with efficacy. Eventually, Mr. Q is titrated to a clozapine dose of 400 mg/d at bedtime.

While receiving clozapine for 4 weeks, Mr. Q’s psychotic symptoms resolve, and he is scheduled for follow-up in the outpatient clozapine clinic.

Related Resources

• De Leon J. A critical commentary on the 2017 AGNP consensus guidelines for therapeutic drug monitoring in neuropsychopharmacology. Pharmacopsychiatry. 2018; 51(1-02):63-68.
• Meyer JM. Is monitoring of plasma antipsychotic levels useful? Current Psychiatry. 2015;14(11):16,19-20.

Drug Brand Names

Aripiprazole • Abilify
Clozapine • Clozaril
Fluphenazine • Prolixin
Haloperidol • Haldol
Lithium • Eskalith, Lithobid
Lurasidone • Latuda
Olanzapine • Zyprexa
Perphenazine • Trilafon
Quetiapine • Seroquel
Risperidone • Risperdal
Valproic acid • Depakene, Depakote
Ziprasidone • Geodon

CASE Worsening mood symptoms and suicidal ideation

On a recent visit to the oncology clinic, where she has been receiving treatment for breast cancer for 11 months, Mrs. L, age 46, reports the abrupt onset of sadness, irritability, difficulty sleeping, and negative self-thoughts.

Eleven months ago, Mrs. L was diagnosed with invasive lobular carcinoma of the right breast that was classified as T2N0MX, representing relatively early-stage disease. Shortly after her diagnosis, Mrs. L completed 4 cycles of neoadjuvant chemotherapy with doxorubicin and cyclophosphamide, followed by treatment with trastuzumab. Subsequently, she underwent a right segmental mastectomy with bilateral mastopexy and radiation therapy. Recently, Mrs. L’s oncology team prescribed tamoxifen, 20 mg/d, and trastuzumab, 420 mg IV every 3 weeks; however, within 3 weeks after starting tamoxifen, Mrs. L’s mood symptoms worsened to the point where she says she is considering suicide—with a plan to use her husband’s gun to kill herself.

Mrs. L has no other pertinent medical history and no reported history of psychiatric disease.

The primary oncology team discontinues tamoxifen (after 5 weeks of treatment) and refers Mrs. L to psychiatry for further mood evaluation.

[polldaddy:10497042]

The authors’ observations

The prevalence of depression is higher in patients with cancer than in the general population.¹ The etiology of depression is often multifactorial.² In Mrs. L’s case, we hypothesized that the possible cause of her depressive symptoms included concerns about her self-image after mastectomy and the adverse effects of chemotherapy and tamoxifen.

Among these possible causes, estrogen level is particularly important. Estrogen affects the brain in numerous ways, including by modulating different neurotransmitters,^3,4 regulating neuroplasticity, providing neuroprotection by preventing formation of oxidative free radicals and of beta amyloid, and possibly avoiding inflammation. From a behavioral standpoint, estrogen acts as an antidepressant while enhancing memory and modulating maternal behavior.⁴ Therefore, decreased estrogen levels could result in depression and other neuro­psychiatric problems. This is illustrated in Mrs. L’s case, where tamoxifen administered after breast cancer treatment coincided with the abrupt onset of depression with suicidal ideation.

Depression in patients receiving tamoxifen might be explained by the fact that tamoxifen is a selective estrogen receptor blocker with dual properties. Specifically, while it has antagonistic action in breast tissue, diminishing the growth-promoting action of estrogen on breast cancer cells, it additionally crosses the blood-brain barrier, so it may block the neuroprotective action of estrogen in the brain.

EXAMINATION Improvement in depression but slightly anxious

During her psychiatric examination, Mrs. L is fairly well-groomed and cooperative. Her speech is normal, thought process is organized, and she has fair insight into her medical situation, with fair judgment. She is alert, attentive, and oriented to time, place, as well as person. She confirms that she has no prior psychiatric history, including no prior suicide attempts. She lives with her husband, who has been supportive. Mrs. L has no children, and she continues to work.

Continue to: Mrs. L reports that per her oncology...

Mrs. L reports that per her oncology team’s instruction, she has not taken tamoxifen for almost 1 week, and notes improvement in her mood. She describes her mood as “fine now,” but appears slightly anxious. She adamantly denies suicidal ideation since stopping tamoxifen; however, she confirms that prior to stopping tamoxifen, she experienced low mood, suicidal thoughts, and a decreased interest in activities. Mrs. L’s Patient Health Questionnaire–9 score is 13, indicating moderate depression. She says she is constantly preoccupied with thoughts about the adverse effects of hormone therapy, and specifically about the oncology team’s suggestion of a retrial of tamoxifen. Due to her constant worry, she has difficulty relaxing; her Generalized Anxiety Disorder–7 item scale score is 12, indicating moderate anxiety. She has a history of cigarette smoking but stopped after her breast cancer diagnosis. She also reports gaining weight since beginning cancer treatment (body mass index: 28.0 kg/m²) and experiencing breast pain.

Mrs. L’s vital signs are normal. Results of her laboratory workup reveal a thyroid-stimulating hormone level of 1.40 µU/mL (reference range: 0.27 to 4.20 µU/mL); a follicle-stimulating hormone (FSH) level of 78.4 mIU/mL (postmenopausal reference range: 25.8 to 134.8 mIU/mL); and an estradiol level of <12.0 pg/mL (postmenopausal range: <55 pg/mL).

The authors’ observations

Studies investigating the effects of tamoxifen on mood have produced varying results (Table^5-16). Some researchers have found a significant relationship between depression and tamoxifen in patients with breast cancer. In a case-control study, 42 postmenopausal women with breast cancer who received tamoxifen reported statistically significant elevated depression scores.⁵ Similarly, in a prospective trial that assessed mood symptoms in 21 pre- and postmenopausal women who developed estrogen deficiency during breast cancer treatment (including treatment with tamoxifen and chemotherapy), 38% of patients met the criteria for major depressive disorder (MDD) in the first 6 months of treatment. Sixty-six percent of these patients were postmenopausal, and 38% were premenopausal. Twenty-five percent of the premenopausal women who experienced MDD symptoms had been treated with tamoxifen and chemotherapy.⁶

In a larger prospective trial (N = 257), an oncologist assessed mood symptoms in 2 groups of patients with breast cancer: individuals who received tamoxifen, and those who did not receive tamoxifen.⁷ They found that 15% of patients who received tamoxifen experienced depression, compared with 3% of patients who did not receive tamoxifen; this difference was statistically significant.⁷ Overall, 31% of the patients had “significant depression” and 27% discontinued tamoxifen because of adverse effects.⁷ There have been 2 case reports of tamoxifen use and severe depression in patients with no prior psychiatry history^8,9 and 3 case reports of tamoxifen use and severe depression in patients who had a psychiatric history.^10-12

One study that examined 24 men with breast cancer found that 62.5% of these patients experienced adverse effects related to tamoxifen, and 25% discontinued tamoxifen because of its adverse effects.¹³ Among the various adverse effects related to tamoxifen, mood alteration was reported in 20.8% of cases, and depressed feelings were reported in 16.6%.¹³

Continue to: Despite the evidence...

Despite this evidence, other studies have not found an association between tamoxifen and depressed mood in patients with breast cancer. One group of researchers who assessed various symptoms self-reported by postmenopausal women who were breast cancer survivors found that the depression scores were not significant.¹⁴ A retrospective cohort study assessed the onset of depression in patients with breast cancer with positive hormone receptor status (who received tamoxifen) vs negative hormone receptor status (who did not receive tamoxifen). These researchers did not find a statistically significant hazard ratio for “new-onset depression.”¹⁵ Unfortunately, the criteria for “new-onset depression” used in this study was the diagnosis of depression or use of an antidepressant given or ordered by a clinician, which is not a sensitive assessment of depressed mood.¹⁵

A multicenter randomized, placebo-controlled trial (the National Surgical Adjuvant Breast and Bowel Project) assessed the incidence of negative health outcomes, including depression, in a secondary outcome analysis.¹⁶ These researchers did not find a statistically significantassociation between tamoxifen and depression. However, in this study, assessment of depression was based on self-report using the Center of Epidemiologic Studies Depression (CES-D) scale, which does not clinically categorize depression. Furthermore, these researchers strongly recommended screening for mood disorders in routine clinical practice. In this study, 3 women completed suicide, 2 of whom were in the tamoxifen arm.¹⁶

[polldaddy:10497045]

The authors’ observations

Tamoxifen is a prodrug that converts to the active metabolite, endoxifen, via cytochrome P450 2D6 (CYP2D6) activity. Antidepressants with strong 2D6-inhibiting properties, such as fluoxetine, duloxetine, bupropion, and paroxetine, should be avoided in patients receiving tamoxifen because they interfere with the formation of the active metabolite and could reduce the effectiveness of tamoxifen and its ability to reduce the risk of cancer recurrence.¹⁷ Antidepressants can help treat psychological distress, especially depression, which is common in patients with cancer, and vasomotor symptoms, which may impair quality of life and adherence to long-term endocrine therapy. Because tamoxifen can decrease cancer recurrence and associated mortality,¹⁸ adherence with treatment is crucial.

TREATMENT Starting an antidepressant

The psychiatry team initiates venlafaxine, 37.5 mg/d, to treat Mrs. L’s anxiety and help prevent the recurrence of severe depression. They prescribe venlafaxine because they anticipate that, based on Mrs. L’s age, the oncology team might reconsider treatment with tamoxifen. Venlafaxine is preferred because it has a favorable pharmacodynamic profile and does not interfere with the metabolism of tamoxifen, as is the case with many selective serotonin reuptake inhibitors.¹⁷

Although Mrs. L’s depression had abated once she stopped receiving tamoxifen, she continues to experience anxiety and tearfulness, primarily due to fear of adverse effects of hormone therapy, and due to family as well as work stressors. Therefore, venlafaxine is gradually titrated up to 150 mg/d.

Continue to: The oncology team proposes...

The oncology team proposes a trial of leuprolide, a gonadotropin-releasing hormone agonist that downregulates pituitary receptors, subsequently suppressing female reproductive hormones, which in turn stops the ovaries from producing estrogen so there is a minimal amount of estrogen to promote the growth of estrogen–receptor-positive breast cancer. Mrs. L declines this agent because she is concerned that she will gain weight. Instead, Mrs. L expresses interest in undergoing an oophorectomy to reduce her estrogen level. In the meantime, based on her reproductive hormone levels (FSH and estradiol levels) which are indicative of postmenopausal status, the oncology team prescribes the aromatase inhibitor (AI) exemestane 25 mg/d. The AI helps to decrease the amount of estrogen the body makes peripherally, which is the main source of estrogen in postmenopausal women.

The authors’ observations

Estrogen originates in the ovaries in premenopausal women; it is also produced by peripheral conversion of androgens to estrogen in adipose tissues and muscle in postmenopausal women.¹⁹ Aromatase inhibitors block the enzyme aromatase that converts androgen to estrogen, which leads to estrogen deficiency in postmenopausal women and possibly to neuropsychiatric effects.¹⁹

The results of studies assessing the adverse psychiatric effects of AIs are mixed. When the results of studies evaluating tamoxifen are compared with those evaluating AIs, overall patients who received AIs had less severe or less frequent mood symptoms. One possible explanation could be that AIs are relatively new compared with tamoxifen. Second, AIs are more commonly used in postmenopausal women with breast cancer, and these patients’ overall estrogen level is significantly lower than that of premenopausal women with breast cancer. Therefore, the degree of hormone fluctuation is less intense in postmenopausal breast cancer survivors.

OUTCOME

After starting exemestane, and while still receiving venlafaxine, Mrs. L no longer experiences severe depressive symptoms. After 8 months, venlafaxine is discontinued. She continues to deny depressive symptoms but has intermittent anxiety, which she is able to manage without psychiatric medication. She continues to remain adherent with ongoing exemestane treatment, with no evidence of disease progression or recurrence.

The authors’ observations

For patients with estrogen-positive breast cancer, the decision to discontinue tamoxifen because of unacceptable adverse effects is an important one because it may increase the risk of cancer recurrence. Psychiatrists have an important role in supporting the patient through this process, helping patients understand alternatives, and working with the oncology team to formulate a plan that is acceptable to everyone.

Continue to: Bottom Line

For patients with estrogen–positive breast cancer, anti-estrogen treatment can reduce the risk of cancer recurrence. However, it can cause adverse effects, including depression, that might impair quality of life and treatment adherence. For patients with severe depression, stopping estrogen blockers may be warranted. Initiating an antidepressant that does not interfere with the metabolism of tamoxifen may help treat depression and vasomotor symptoms.

Related Resource

• Agarwala P. Tailoring depression treatment for women with breast cancer. Current Psychiatry. 2010;9(11):39-40,45-46,48-49.

Drug Brand Names

Agomelatine • Valdoxan
Bupropion • Wellbutrin, Zyban
Cyclophosphamide • Cytoxan
Doxorubicin • Adriamycin
Duloxetine • Cymbalta
Exemestane • Aromasin
Fluoxetine • Prozac
Leuprolide • Eligard, Lupron
Milnacipran • Savella
Paroxetine • Paxil
Sertraline • Zoloft
Tamoxifen • Soltamox
Trastuzumab • Herceptin
Venlafaxine • Effexor

CASE Worsening mood symptoms and suicidal ideation

On a recent visit to the oncology clinic, where she has been receiving treatment for breast cancer for 11 months, Mrs. L, age 46, reports the abrupt onset of sadness, irritability, difficulty sleeping, and negative self-thoughts.

Eleven months ago, Mrs. L was diagnosed with invasive lobular carcinoma of the right breast that was classified as T2N0MX, representing relatively early-stage disease. Shortly after her diagnosis, Mrs. L completed 4 cycles of neoadjuvant chemotherapy with doxorubicin and cyclophosphamide, followed by treatment with trastuzumab. Subsequently, she underwent a right segmental mastectomy with bilateral mastopexy and radiation therapy. Recently, Mrs. L’s oncology team prescribed tamoxifen, 20 mg/d, and trastuzumab, 420 mg IV every 3 weeks; however, within 3 weeks after starting tamoxifen, Mrs. L’s mood symptoms worsened to the point where she says she is considering suicide—with a plan to use her husband’s gun to kill herself.

Mrs. L has no other pertinent medical history and no reported history of psychiatric disease.

The primary oncology team discontinues tamoxifen (after 5 weeks of treatment) and refers Mrs. L to psychiatry for further mood evaluation.

[polldaddy:10497042]

The authors’ observations

The prevalence of depression is higher in patients with cancer than in the general population.¹ The etiology of depression is often multifactorial.² In Mrs. L’s case, we hypothesized that the possible cause of her depressive symptoms included concerns about her self-image after mastectomy and the adverse effects of chemotherapy and tamoxifen.

Among these possible causes, estrogen level is particularly important. Estrogen affects the brain in numerous ways, including by modulating different neurotransmitters,^3,4 regulating neuroplasticity, providing neuroprotection by preventing formation of oxidative free radicals and of beta amyloid, and possibly avoiding inflammation. From a behavioral standpoint, estrogen acts as an antidepressant while enhancing memory and modulating maternal behavior.⁴ Therefore, decreased estrogen levels could result in depression and other neuro­psychiatric problems. This is illustrated in Mrs. L’s case, where tamoxifen administered after breast cancer treatment coincided with the abrupt onset of depression with suicidal ideation.

Depression in patients receiving tamoxifen might be explained by the fact that tamoxifen is a selective estrogen receptor blocker with dual properties. Specifically, while it has antagonistic action in breast tissue, diminishing the growth-promoting action of estrogen on breast cancer cells, it additionally crosses the blood-brain barrier, so it may block the neuroprotective action of estrogen in the brain.

EXAMINATION Improvement in depression but slightly anxious

During her psychiatric examination, Mrs. L is fairly well-groomed and cooperative. Her speech is normal, thought process is organized, and she has fair insight into her medical situation, with fair judgment. She is alert, attentive, and oriented to time, place, as well as person. She confirms that she has no prior psychiatric history, including no prior suicide attempts. She lives with her husband, who has been supportive. Mrs. L has no children, and she continues to work.

Continue to: Mrs. L reports that per her oncology...

Mrs. L reports that per her oncology team’s instruction, she has not taken tamoxifen for almost 1 week, and notes improvement in her mood. She describes her mood as “fine now,” but appears slightly anxious. She adamantly denies suicidal ideation since stopping tamoxifen; however, she confirms that prior to stopping tamoxifen, she experienced low mood, suicidal thoughts, and a decreased interest in activities. Mrs. L’s Patient Health Questionnaire–9 score is 13, indicating moderate depression. She says she is constantly preoccupied with thoughts about the adverse effects of hormone therapy, and specifically about the oncology team’s suggestion of a retrial of tamoxifen. Due to her constant worry, she has difficulty relaxing; her Generalized Anxiety Disorder–7 item scale score is 12, indicating moderate anxiety. She has a history of cigarette smoking but stopped after her breast cancer diagnosis. She also reports gaining weight since beginning cancer treatment (body mass index: 28.0 kg/m²) and experiencing breast pain.

Mrs. L’s vital signs are normal. Results of her laboratory workup reveal a thyroid-stimulating hormone level of 1.40 µU/mL (reference range: 0.27 to 4.20 µU/mL); a follicle-stimulating hormone (FSH) level of 78.4 mIU/mL (postmenopausal reference range: 25.8 to 134.8 mIU/mL); and an estradiol level of <12.0 pg/mL (postmenopausal range: <55 pg/mL).

The authors’ observations

Studies investigating the effects of tamoxifen on mood have produced varying results (Table^5-16). Some researchers have found a significant relationship between depression and tamoxifen in patients with breast cancer. In a case-control study, 42 postmenopausal women with breast cancer who received tamoxifen reported statistically significant elevated depression scores.⁵ Similarly, in a prospective trial that assessed mood symptoms in 21 pre- and postmenopausal women who developed estrogen deficiency during breast cancer treatment (including treatment with tamoxifen and chemotherapy), 38% of patients met the criteria for major depressive disorder (MDD) in the first 6 months of treatment. Sixty-six percent of these patients were postmenopausal, and 38% were premenopausal. Twenty-five percent of the premenopausal women who experienced MDD symptoms had been treated with tamoxifen and chemotherapy.⁶

In a larger prospective trial (N = 257), an oncologist assessed mood symptoms in 2 groups of patients with breast cancer: individuals who received tamoxifen, and those who did not receive tamoxifen.⁷ They found that 15% of patients who received tamoxifen experienced depression, compared with 3% of patients who did not receive tamoxifen; this difference was statistically significant.⁷ Overall, 31% of the patients had “significant depression” and 27% discontinued tamoxifen because of adverse effects.⁷ There have been 2 case reports of tamoxifen use and severe depression in patients with no prior psychiatry history^8,9 and 3 case reports of tamoxifen use and severe depression in patients who had a psychiatric history.^10-12

One study that examined 24 men with breast cancer found that 62.5% of these patients experienced adverse effects related to tamoxifen, and 25% discontinued tamoxifen because of its adverse effects.¹³ Among the various adverse effects related to tamoxifen, mood alteration was reported in 20.8% of cases, and depressed feelings were reported in 16.6%.¹³

Continue to: Despite the evidence...

Despite this evidence, other studies have not found an association between tamoxifen and depressed mood in patients with breast cancer. One group of researchers who assessed various symptoms self-reported by postmenopausal women who were breast cancer survivors found that the depression scores were not significant.¹⁴ A retrospective cohort study assessed the onset of depression in patients with breast cancer with positive hormone receptor status (who received tamoxifen) vs negative hormone receptor status (who did not receive tamoxifen). These researchers did not find a statistically significant hazard ratio for “new-onset depression.”¹⁵ Unfortunately, the criteria for “new-onset depression” used in this study was the diagnosis of depression or use of an antidepressant given or ordered by a clinician, which is not a sensitive assessment of depressed mood.¹⁵

A multicenter randomized, placebo-controlled trial (the National Surgical Adjuvant Breast and Bowel Project) assessed the incidence of negative health outcomes, including depression, in a secondary outcome analysis.¹⁶ These researchers did not find a statistically significantassociation between tamoxifen and depression. However, in this study, assessment of depression was based on self-report using the Center of Epidemiologic Studies Depression (CES-D) scale, which does not clinically categorize depression. Furthermore, these researchers strongly recommended screening for mood disorders in routine clinical practice. In this study, 3 women completed suicide, 2 of whom were in the tamoxifen arm.¹⁶

[polldaddy:10497045]

The authors’ observations

Tamoxifen is a prodrug that converts to the active metabolite, endoxifen, via cytochrome P450 2D6 (CYP2D6) activity. Antidepressants with strong 2D6-inhibiting properties, such as fluoxetine, duloxetine, bupropion, and paroxetine, should be avoided in patients receiving tamoxifen because they interfere with the formation of the active metabolite and could reduce the effectiveness of tamoxifen and its ability to reduce the risk of cancer recurrence.¹⁷ Antidepressants can help treat psychological distress, especially depression, which is common in patients with cancer, and vasomotor symptoms, which may impair quality of life and adherence to long-term endocrine therapy. Because tamoxifen can decrease cancer recurrence and associated mortality,¹⁸ adherence with treatment is crucial.

TREATMENT Starting an antidepressant

The psychiatry team initiates venlafaxine, 37.5 mg/d, to treat Mrs. L’s anxiety and help prevent the recurrence of severe depression. They prescribe venlafaxine because they anticipate that, based on Mrs. L’s age, the oncology team might reconsider treatment with tamoxifen. Venlafaxine is preferred because it has a favorable pharmacodynamic profile and does not interfere with the metabolism of tamoxifen, as is the case with many selective serotonin reuptake inhibitors.¹⁷

Although Mrs. L’s depression had abated once she stopped receiving tamoxifen, she continues to experience anxiety and tearfulness, primarily due to fear of adverse effects of hormone therapy, and due to family as well as work stressors. Therefore, venlafaxine is gradually titrated up to 150 mg/d.

Continue to: The oncology team proposes...

The oncology team proposes a trial of leuprolide, a gonadotropin-releasing hormone agonist that downregulates pituitary receptors, subsequently suppressing female reproductive hormones, which in turn stops the ovaries from producing estrogen so there is a minimal amount of estrogen to promote the growth of estrogen–receptor-positive breast cancer. Mrs. L declines this agent because she is concerned that she will gain weight. Instead, Mrs. L expresses interest in undergoing an oophorectomy to reduce her estrogen level. In the meantime, based on her reproductive hormone levels (FSH and estradiol levels) which are indicative of postmenopausal status, the oncology team prescribes the aromatase inhibitor (AI) exemestane 25 mg/d. The AI helps to decrease the amount of estrogen the body makes peripherally, which is the main source of estrogen in postmenopausal women.

The authors’ observations

Estrogen originates in the ovaries in premenopausal women; it is also produced by peripheral conversion of androgens to estrogen in adipose tissues and muscle in postmenopausal women.¹⁹ Aromatase inhibitors block the enzyme aromatase that converts androgen to estrogen, which leads to estrogen deficiency in postmenopausal women and possibly to neuropsychiatric effects.¹⁹

The results of studies assessing the adverse psychiatric effects of AIs are mixed. When the results of studies evaluating tamoxifen are compared with those evaluating AIs, overall patients who received AIs had less severe or less frequent mood symptoms. One possible explanation could be that AIs are relatively new compared with tamoxifen. Second, AIs are more commonly used in postmenopausal women with breast cancer, and these patients’ overall estrogen level is significantly lower than that of premenopausal women with breast cancer. Therefore, the degree of hormone fluctuation is less intense in postmenopausal breast cancer survivors.

OUTCOME

After starting exemestane, and while still receiving venlafaxine, Mrs. L no longer experiences severe depressive symptoms. After 8 months, venlafaxine is discontinued. She continues to deny depressive symptoms but has intermittent anxiety, which she is able to manage without psychiatric medication. She continues to remain adherent with ongoing exemestane treatment, with no evidence of disease progression or recurrence.

The authors’ observations

For patients with estrogen-positive breast cancer, the decision to discontinue tamoxifen because of unacceptable adverse effects is an important one because it may increase the risk of cancer recurrence. Psychiatrists have an important role in supporting the patient through this process, helping patients understand alternatives, and working with the oncology team to formulate a plan that is acceptable to everyone.

Continue to: Bottom Line

For patients with estrogen–positive breast cancer, anti-estrogen treatment can reduce the risk of cancer recurrence. However, it can cause adverse effects, including depression, that might impair quality of life and treatment adherence. For patients with severe depression, stopping estrogen blockers may be warranted. Initiating an antidepressant that does not interfere with the metabolism of tamoxifen may help treat depression and vasomotor symptoms.

Related Resource

• Agarwala P. Tailoring depression treatment for women with breast cancer. Current Psychiatry. 2010;9(11):39-40,45-46,48-49.

Drug Brand Names

Agomelatine • Valdoxan
Bupropion • Wellbutrin, Zyban
Cyclophosphamide • Cytoxan
Doxorubicin • Adriamycin
Duloxetine • Cymbalta
Exemestane • Aromasin
Fluoxetine • Prozac
Leuprolide • Eligard, Lupron
Milnacipran • Savella
Paroxetine • Paxil
Sertraline • Zoloft
Tamoxifen • Soltamox
Trastuzumab • Herceptin
Venlafaxine • Effexor

CASE Worsening mood symptoms and suicidal ideation

On a recent visit to the oncology clinic, where she has been receiving treatment for breast cancer for 11 months, Mrs. L, age 46, reports the abrupt onset of sadness, irritability, difficulty sleeping, and negative self-thoughts.

Eleven months ago, Mrs. L was diagnosed with invasive lobular carcinoma of the right breast that was classified as T2N0MX, representing relatively early-stage disease. Shortly after her diagnosis, Mrs. L completed 4 cycles of neoadjuvant chemotherapy with doxorubicin and cyclophosphamide, followed by treatment with trastuzumab. Subsequently, she underwent a right segmental mastectomy with bilateral mastopexy and radiation therapy. Recently, Mrs. L’s oncology team prescribed tamoxifen, 20 mg/d, and trastuzumab, 420 mg IV every 3 weeks; however, within 3 weeks after starting tamoxifen, Mrs. L’s mood symptoms worsened to the point where she says she is considering suicide—with a plan to use her husband’s gun to kill herself.

Mrs. L has no other pertinent medical history and no reported history of psychiatric disease.

The primary oncology team discontinues tamoxifen (after 5 weeks of treatment) and refers Mrs. L to psychiatry for further mood evaluation.

[polldaddy:10497042]

The authors’ observations

The prevalence of depression is higher in patients with cancer than in the general population.¹ The etiology of depression is often multifactorial.² In Mrs. L’s case, we hypothesized that the possible cause of her depressive symptoms included concerns about her self-image after mastectomy and the adverse effects of chemotherapy and tamoxifen.

Among these possible causes, estrogen level is particularly important. Estrogen affects the brain in numerous ways, including by modulating different neurotransmitters,^3,4 regulating neuroplasticity, providing neuroprotection by preventing formation of oxidative free radicals and of beta amyloid, and possibly avoiding inflammation. From a behavioral standpoint, estrogen acts as an antidepressant while enhancing memory and modulating maternal behavior.⁴ Therefore, decreased estrogen levels could result in depression and other neuro­psychiatric problems. This is illustrated in Mrs. L’s case, where tamoxifen administered after breast cancer treatment coincided with the abrupt onset of depression with suicidal ideation.

Depression in patients receiving tamoxifen might be explained by the fact that tamoxifen is a selective estrogen receptor blocker with dual properties. Specifically, while it has antagonistic action in breast tissue, diminishing the growth-promoting action of estrogen on breast cancer cells, it additionally crosses the blood-brain barrier, so it may block the neuroprotective action of estrogen in the brain.

EXAMINATION Improvement in depression but slightly anxious

During her psychiatric examination, Mrs. L is fairly well-groomed and cooperative. Her speech is normal, thought process is organized, and she has fair insight into her medical situation, with fair judgment. She is alert, attentive, and oriented to time, place, as well as person. She confirms that she has no prior psychiatric history, including no prior suicide attempts. She lives with her husband, who has been supportive. Mrs. L has no children, and she continues to work.

Continue to: Mrs. L reports that per her oncology...

Mrs. L reports that per her oncology team’s instruction, she has not taken tamoxifen for almost 1 week, and notes improvement in her mood. She describes her mood as “fine now,” but appears slightly anxious. She adamantly denies suicidal ideation since stopping tamoxifen; however, she confirms that prior to stopping tamoxifen, she experienced low mood, suicidal thoughts, and a decreased interest in activities. Mrs. L’s Patient Health Questionnaire–9 score is 13, indicating moderate depression. She says she is constantly preoccupied with thoughts about the adverse effects of hormone therapy, and specifically about the oncology team’s suggestion of a retrial of tamoxifen. Due to her constant worry, she has difficulty relaxing; her Generalized Anxiety Disorder–7 item scale score is 12, indicating moderate anxiety. She has a history of cigarette smoking but stopped after her breast cancer diagnosis. She also reports gaining weight since beginning cancer treatment (body mass index: 28.0 kg/m²) and experiencing breast pain.

Mrs. L’s vital signs are normal. Results of her laboratory workup reveal a thyroid-stimulating hormone level of 1.40 µU/mL (reference range: 0.27 to 4.20 µU/mL); a follicle-stimulating hormone (FSH) level of 78.4 mIU/mL (postmenopausal reference range: 25.8 to 134.8 mIU/mL); and an estradiol level of <12.0 pg/mL (postmenopausal range: <55 pg/mL).

The authors’ observations

Studies investigating the effects of tamoxifen on mood have produced varying results (Table^5-16). Some researchers have found a significant relationship between depression and tamoxifen in patients with breast cancer. In a case-control study, 42 postmenopausal women with breast cancer who received tamoxifen reported statistically significant elevated depression scores.⁵ Similarly, in a prospective trial that assessed mood symptoms in 21 pre- and postmenopausal women who developed estrogen deficiency during breast cancer treatment (including treatment with tamoxifen and chemotherapy), 38% of patients met the criteria for major depressive disorder (MDD) in the first 6 months of treatment. Sixty-six percent of these patients were postmenopausal, and 38% were premenopausal. Twenty-five percent of the premenopausal women who experienced MDD symptoms had been treated with tamoxifen and chemotherapy.⁶

In a larger prospective trial (N = 257), an oncologist assessed mood symptoms in 2 groups of patients with breast cancer: individuals who received tamoxifen, and those who did not receive tamoxifen.⁷ They found that 15% of patients who received tamoxifen experienced depression, compared with 3% of patients who did not receive tamoxifen; this difference was statistically significant.⁷ Overall, 31% of the patients had “significant depression” and 27% discontinued tamoxifen because of adverse effects.⁷ There have been 2 case reports of tamoxifen use and severe depression in patients with no prior psychiatry history^8,9 and 3 case reports of tamoxifen use and severe depression in patients who had a psychiatric history.^10-12

One study that examined 24 men with breast cancer found that 62.5% of these patients experienced adverse effects related to tamoxifen, and 25% discontinued tamoxifen because of its adverse effects.¹³ Among the various adverse effects related to tamoxifen, mood alteration was reported in 20.8% of cases, and depressed feelings were reported in 16.6%.¹³

Continue to: Despite the evidence...

Despite this evidence, other studies have not found an association between tamoxifen and depressed mood in patients with breast cancer. One group of researchers who assessed various symptoms self-reported by postmenopausal women who were breast cancer survivors found that the depression scores were not significant.¹⁴ A retrospective cohort study assessed the onset of depression in patients with breast cancer with positive hormone receptor status (who received tamoxifen) vs negative hormone receptor status (who did not receive tamoxifen). These researchers did not find a statistically significant hazard ratio for “new-onset depression.”¹⁵ Unfortunately, the criteria for “new-onset depression” used in this study was the diagnosis of depression or use of an antidepressant given or ordered by a clinician, which is not a sensitive assessment of depressed mood.¹⁵

A multicenter randomized, placebo-controlled trial (the National Surgical Adjuvant Breast and Bowel Project) assessed the incidence of negative health outcomes, including depression, in a secondary outcome analysis.¹⁶ These researchers did not find a statistically significantassociation between tamoxifen and depression. However, in this study, assessment of depression was based on self-report using the Center of Epidemiologic Studies Depression (CES-D) scale, which does not clinically categorize depression. Furthermore, these researchers strongly recommended screening for mood disorders in routine clinical practice. In this study, 3 women completed suicide, 2 of whom were in the tamoxifen arm.¹⁶

[polldaddy:10497045]

The authors’ observations

Tamoxifen is a prodrug that converts to the active metabolite, endoxifen, via cytochrome P450 2D6 (CYP2D6) activity. Antidepressants with strong 2D6-inhibiting properties, such as fluoxetine, duloxetine, bupropion, and paroxetine, should be avoided in patients receiving tamoxifen because they interfere with the formation of the active metabolite and could reduce the effectiveness of tamoxifen and its ability to reduce the risk of cancer recurrence.¹⁷ Antidepressants can help treat psychological distress, especially depression, which is common in patients with cancer, and vasomotor symptoms, which may impair quality of life and adherence to long-term endocrine therapy. Because tamoxifen can decrease cancer recurrence and associated mortality,¹⁸ adherence with treatment is crucial.

TREATMENT Starting an antidepressant

The psychiatry team initiates venlafaxine, 37.5 mg/d, to treat Mrs. L’s anxiety and help prevent the recurrence of severe depression. They prescribe venlafaxine because they anticipate that, based on Mrs. L’s age, the oncology team might reconsider treatment with tamoxifen. Venlafaxine is preferred because it has a favorable pharmacodynamic profile and does not interfere with the metabolism of tamoxifen, as is the case with many selective serotonin reuptake inhibitors.¹⁷

Although Mrs. L’s depression had abated once she stopped receiving tamoxifen, she continues to experience anxiety and tearfulness, primarily due to fear of adverse effects of hormone therapy, and due to family as well as work stressors. Therefore, venlafaxine is gradually titrated up to 150 mg/d.

Continue to: The oncology team proposes...

The oncology team proposes a trial of leuprolide, a gonadotropin-releasing hormone agonist that downregulates pituitary receptors, subsequently suppressing female reproductive hormones, which in turn stops the ovaries from producing estrogen so there is a minimal amount of estrogen to promote the growth of estrogen–receptor-positive breast cancer. Mrs. L declines this agent because she is concerned that she will gain weight. Instead, Mrs. L expresses interest in undergoing an oophorectomy to reduce her estrogen level. In the meantime, based on her reproductive hormone levels (FSH and estradiol levels) which are indicative of postmenopausal status, the oncology team prescribes the aromatase inhibitor (AI) exemestane 25 mg/d. The AI helps to decrease the amount of estrogen the body makes peripherally, which is the main source of estrogen in postmenopausal women.

The authors’ observations

Estrogen originates in the ovaries in premenopausal women; it is also produced by peripheral conversion of androgens to estrogen in adipose tissues and muscle in postmenopausal women.¹⁹ Aromatase inhibitors block the enzyme aromatase that converts androgen to estrogen, which leads to estrogen deficiency in postmenopausal women and possibly to neuropsychiatric effects.¹⁹

The results of studies assessing the adverse psychiatric effects of AIs are mixed. When the results of studies evaluating tamoxifen are compared with those evaluating AIs, overall patients who received AIs had less severe or less frequent mood symptoms. One possible explanation could be that AIs are relatively new compared with tamoxifen. Second, AIs are more commonly used in postmenopausal women with breast cancer, and these patients’ overall estrogen level is significantly lower than that of premenopausal women with breast cancer. Therefore, the degree of hormone fluctuation is less intense in postmenopausal breast cancer survivors.

OUTCOME

After starting exemestane, and while still receiving venlafaxine, Mrs. L no longer experiences severe depressive symptoms. After 8 months, venlafaxine is discontinued. She continues to deny depressive symptoms but has intermittent anxiety, which she is able to manage without psychiatric medication. She continues to remain adherent with ongoing exemestane treatment, with no evidence of disease progression or recurrence.

The authors’ observations

For patients with estrogen-positive breast cancer, the decision to discontinue tamoxifen because of unacceptable adverse effects is an important one because it may increase the risk of cancer recurrence. Psychiatrists have an important role in supporting the patient through this process, helping patients understand alternatives, and working with the oncology team to formulate a plan that is acceptable to everyone.

Continue to: Bottom Line

For patients with estrogen–positive breast cancer, anti-estrogen treatment can reduce the risk of cancer recurrence. However, it can cause adverse effects, including depression, that might impair quality of life and treatment adherence. For patients with severe depression, stopping estrogen blockers may be warranted. Initiating an antidepressant that does not interfere with the metabolism of tamoxifen may help treat depression and vasomotor symptoms.

Related Resource

• Agarwala P. Tailoring depression treatment for women with breast cancer. Current Psychiatry. 2010;9(11):39-40,45-46,48-49.

Drug Brand Names

Agomelatine • Valdoxan
Bupropion • Wellbutrin, Zyban
Cyclophosphamide • Cytoxan
Doxorubicin • Adriamycin
Duloxetine • Cymbalta
Exemestane • Aromasin
Fluoxetine • Prozac
Leuprolide • Eligard, Lupron
Milnacipran • Savella
Paroxetine • Paxil
Sertraline • Zoloft
Tamoxifen • Soltamox
Trastuzumab • Herceptin
Venlafaxine • Effexor

Mnemonics and acronyms are part of our daily lives, helping us to memorize and retain clinical information. They play an invaluable role in medical school because they can help students recall vast amounts of information in a moment’s notice, such as psychiatric conditions to consider during a “review of systems.”

Most medical students are trained to conduct a review of systems as a standard approach when a thorough medical history is indicated. Clinicians need to assess all patients for an extremely broad range of syndromes. Because of the extensive comorbidity of many psychiatric disorders, it is important to review the most common conditions before establishing a diagnosis and formulating a treatment plan.¹

For example, a patient presenting with a chief complaint consistent with a depressive disorder may have unipolar depression, bipolar depression, or substance-induced depression (after general medical comorbidity has been excluded). In this scenario, it would be equally important to identify co-occurring conditions, such as an anxiety disorder or psychotic symptoms, because these can have a major impact on treatment and prognosis.

In our work as clinical educators, we have noticed that many students struggle with a review of psychiatric systems during their evaluation of a new patient. Acronyms could serve as a map to guide them during assessments. While these may be most valuable to medical students, they are also helpful for clinicians on the frontline of medical practice (primary care, family practice, OB-GYN) as well as for early-career psychiatrists.

MAPS.EDU

The acronym MAPS.EDU covers several common psychiatric conditions seen in routine practice: Mood disorders, Anxiety disorders, Personality disorders, Schizophrenia and related disorders, Eating disorders, Developmental disorders (eg, attention-deficit/hyperactivity disorder), and substance Use disorders. While not comprehensive, MAPS.EDU can be a quick method to help psychiatrists remember these common conditions.

GAS POPS

Anxiety is a core symptom of several psychiatric disorders. The mnemonic GAS POPS can help clinicians recall disorders to consider when screening patients who report anxiety: Generalized anxiety disorder, Agoraphobia, Social anxiety disorder, Panic disorder, Obsessive-compulsive disorder, Posttraumatic stress disorder (PTSD), and Specific phobias.

AEIOU for PTSD

The diagnostic criteria of PTSD can be memorized by using the acronym AEIOU: Avoidance (of triggers), Exposure (to trauma), Intrusions (reliving phenomena), Outbursts (or other manifestations of hyperarousal), and Unhappiness (negative alterations in mood and cognition).

Mnemonics and acronyms are part of our daily lives, helping us to memorize and retain clinical information. They play an invaluable role in medical school because they can help students recall vast amounts of information in a moment’s notice, such as psychiatric conditions to consider during a “review of systems.”

Most medical students are trained to conduct a review of systems as a standard approach when a thorough medical history is indicated. Clinicians need to assess all patients for an extremely broad range of syndromes. Because of the extensive comorbidity of many psychiatric disorders, it is important to review the most common conditions before establishing a diagnosis and formulating a treatment plan.¹

For example, a patient presenting with a chief complaint consistent with a depressive disorder may have unipolar depression, bipolar depression, or substance-induced depression (after general medical comorbidity has been excluded). In this scenario, it would be equally important to identify co-occurring conditions, such as an anxiety disorder or psychotic symptoms, because these can have a major impact on treatment and prognosis.

In our work as clinical educators, we have noticed that many students struggle with a review of psychiatric systems during their evaluation of a new patient. Acronyms could serve as a map to guide them during assessments. While these may be most valuable to medical students, they are also helpful for clinicians on the frontline of medical practice (primary care, family practice, OB-GYN) as well as for early-career psychiatrists.

MAPS.EDU

The acronym MAPS.EDU covers several common psychiatric conditions seen in routine practice: Mood disorders, Anxiety disorders, Personality disorders, Schizophrenia and related disorders, Eating disorders, Developmental disorders (eg, attention-deficit/hyperactivity disorder), and substance Use disorders. While not comprehensive, MAPS.EDU can be a quick method to help psychiatrists remember these common conditions.

GAS POPS

Anxiety is a core symptom of several psychiatric disorders. The mnemonic GAS POPS can help clinicians recall disorders to consider when screening patients who report anxiety: Generalized anxiety disorder, Agoraphobia, Social anxiety disorder, Panic disorder, Obsessive-compulsive disorder, Posttraumatic stress disorder (PTSD), and Specific phobias.

AEIOU for PTSD

The diagnostic criteria of PTSD can be memorized by using the acronym AEIOU: Avoidance (of triggers), Exposure (to trauma), Intrusions (reliving phenomena), Outbursts (or other manifestations of hyperarousal), and Unhappiness (negative alterations in mood and cognition).

Mnemonics and acronyms are part of our daily lives, helping us to memorize and retain clinical information. They play an invaluable role in medical school because they can help students recall vast amounts of information in a moment’s notice, such as psychiatric conditions to consider during a “review of systems.”

Most medical students are trained to conduct a review of systems as a standard approach when a thorough medical history is indicated. Clinicians need to assess all patients for an extremely broad range of syndromes. Because of the extensive comorbidity of many psychiatric disorders, it is important to review the most common conditions before establishing a diagnosis and formulating a treatment plan.¹

For example, a patient presenting with a chief complaint consistent with a depressive disorder may have unipolar depression, bipolar depression, or substance-induced depression (after general medical comorbidity has been excluded). In this scenario, it would be equally important to identify co-occurring conditions, such as an anxiety disorder or psychotic symptoms, because these can have a major impact on treatment and prognosis.

In our work as clinical educators, we have noticed that many students struggle with a review of psychiatric systems during their evaluation of a new patient. Acronyms could serve as a map to guide them during assessments. While these may be most valuable to medical students, they are also helpful for clinicians on the frontline of medical practice (primary care, family practice, OB-GYN) as well as for early-career psychiatrists.

MAPS.EDU

The acronym MAPS.EDU covers several common psychiatric conditions seen in routine practice: Mood disorders, Anxiety disorders, Personality disorders, Schizophrenia and related disorders, Eating disorders, Developmental disorders (eg, attention-deficit/hyperactivity disorder), and substance Use disorders. While not comprehensive, MAPS.EDU can be a quick method to help psychiatrists remember these common conditions.

GAS POPS

Anxiety is a core symptom of several psychiatric disorders. The mnemonic GAS POPS can help clinicians recall disorders to consider when screening patients who report anxiety: Generalized anxiety disorder, Agoraphobia, Social anxiety disorder, Panic disorder, Obsessive-compulsive disorder, Posttraumatic stress disorder (PTSD), and Specific phobias.

AEIOU for PTSD

The diagnostic criteria of PTSD can be memorized by using the acronym AEIOU: Avoidance (of triggers), Exposure (to trauma), Intrusions (reliving phenomena), Outbursts (or other manifestations of hyperarousal), and Unhappiness (negative alterations in mood and cognition).

From Dell Medical School, The University of Texas at Austin, Austin, TX.

Abstract

• Objective: Satisfaction measures often show substantial ceiling effects. This randomized controlled trial tested the null hypothesis that there is no difference in mean overall satisfaction, ceiling and floor effect, and data distribution between 4 different kinds of single-question scales assessing the helpfulness of a visit. We also hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared with the Net Promoter Scores (NPS).
• Design: Randomized controlled trial.
• Methods: We enrolled 258 adult, English-speaking new and returning patients. Patients were randomly assigned to 1 of 4 different scale types: (1) an 11-point ordinal scale with 5 anchor points; (2) a 5-point Likert scale; (3) a 0-100 visual analogue scale (VAS) electronic slider with 3 anchor points and visible numbers; and (4) a 0-100 VAS with 3 anchor points and no visible numbers. Additionally, patients completed the 2-item Pain Self-Efficacy Questionnaire (PSEQ-2), 5-item Short Health Anxiety Inventory scale (SHAI-5), and Patient-Reported Outcomes Measurement Information System (PROMIS) Depression. We assessed mean and median score, floor and ceiling effect, and skewness and kurtosis for each scale. Spearman correlation tests were used to test correlations between satisfaction and psychological status.
• Results: The nonnumerical 0-100 VAS with 3 anchor points and the 5-point Likert scale had the least ceiling effect (12% and 20%, respectively). The 11-point ordinal scale had skewness and kurtosis closest to a normal distribution (skew = –0.58 and kurtosis = 4.0). Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064). NPS were 35, 16, 67, and 20 for the scales, respectively.
• Conclusion: Single-question measures of satisfaction can be adjusted to limit the ceiling effect. Additional research in this area is warranted.

Keywords: patient satisfaction; floor and ceiling effect; skewness and kurtosis; quality improvement.

Patient satisfaction is an important quality metric that is increasingly being measured, reported, and incentivized. A qualitative study identified 7 themes influencing satisfaction among people visiting an orthopedic surgeon’s office: trust, relatedness, expectations, wait time, visit duration, communication, and empathy.¹ However, another study found that satisfaction and perceived empathy are not associated with wait time or visit duration, but rather with the quality of the visit.² Satisfaction measures that incorporate many of these features in relatively long questionnaires are associated with lower response rates³ and overlap with the factors whose influence on satisfaction one would like to study (eg, perceived empathy or communication effectiveness).⁴ Single- and multiple-question satisfaction scores are prone to a strong right skew, with a substantial ceiling effect.⁵ Ceiling effect occurs when a considerable proportion (about half) of participants select 1 of the top 2 scores (or the maximum score). An ideal scale would measure satisfaction independent from other factors, would use 1 or just a few questions, and would have little or no ceiling effect.

In this randomized controlled trial, we examined whether there were significant differences in mean and median satisfaction, floor and ceiling effect, and data distribution (by looking at skewness and kurtosis) between 4 different kinds of satisfaction scales asking about the helpfulness of a visit. Additionally, we hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared to the Net Promoter Scores (NPS). NPS are commonly used in the service industry to measure customer satisfaction; we are using these scores as a measure of patient satisfaction.

Methods

Study Design

All English-speaking new and return patients ages 18 to 89 years visiting an orthopedic surgeon in 1 of 7 clinics located in a large urban area were considered eligible for this study. Enrollment took place intermittently over a 5-month period. We were granted a waiver of written informed consent. Patients indicated their consent by completing the surveys. Patients were randomly assigned to 1 of the 4 questionnaires containing different scale types using an Excel random-number generator. After the visit, patients were asked to complete the survey. All questionnaires were administered on an encrypted tablet via a HIPAA-compliant, secure web-based application for building and managing online surveys and databases (REDCap; Research Electronic Data Capture).⁶ This study was approved by our Institutional Review Board and is registered on ClinicalTrials.gov (NCT03686735).⁷

Outcome Measures

Study participants were asked to complete questionnaires regarding demographics (sex, age, race/ethnicity, marital status, level of education, work status, insurance status, comorbidities) and to rate satisfaction with their visit on the scale that was randomly assigned to them: (1) an 11-point Likert scale with 5 anchor points and visible numbers; (2) a 5-point Likert scale with 5 anchor points and no visible numbers; (3) a 0-100 VAS with 3 anchor points and visible numbers; (4) a 0-100 VAS with 3 anchor points and no visible numbers (Figure 1). The 4 scales should not differ in time needed to complete them; however, we did not explicitly measure time to completion. Participants also completed measures of psychological aspects of illness. The 2-item Pain Self-Efficacy Questionnaire (PSEQ-2) was used to measure pain self-efficacy, an effective coping strategy for pain.⁸ Higher PSEQ-2 scores indicate a higher level of pain self-efficacy. The 5-item Short Health Anxiety Inventory scale (SHAI-5) was also administered; higher scores on this scale indicate a greater degree of health anxiety.⁹ The Patient-Reported Outcomes Measurement Information System (PROMIS) Depression was used to measure symptoms of depression.¹⁰ Finally, the diagnosis was recorded by the surgeon (not in table).

Statistical Analysis

We reported continuous variables using mean, standard deviation (SD), median, and interquartile range (IQR). Categorical data are presented as frequencies and percentages. We calculated floor and ceiling effect and the skewness and kurtosis of every scale. We scaled every scale to 10 and also standardized every scale. We used the Kruskal–Wallis test to compare differences in satisfaction between the scales; Fisher’s exact test to compare differences in floor and ceiling effect; and Spearman correlation tests to test the correlation between scaled satisfaction scores and psychological status.

Ceiling effects are present when patients select the highest value on a scale rather than a value that reflects their actual feelings about a certain topic. Floor effects are present when patients select the lowest value in a similar fashion. These 2 effects indicate that an independent variable no longer influences the dependent variable being tested. Skewness and kurtosis are rough indicators of a normal distribution of values. Skewness (γ1) is an index of the symmetry of a distribution, with symmetric distributions having a skewness of 0. If skewness has a positive value, it suggests relatively many low values, having a long right tail. Negative skewness suggests relatively many high values, having a long left tail. Kurtosis (γ2) is a measure to describe tailedness of a distribution. Kurtosis of a normal distribution is 3. Negative kurtosis represents little peaked distribution, and positive kurtosis represents more peaked distribution.^11,12 If skewness is 0 and kurtosis is 3, there is a normal, or Gaussian, distribution.

Finally, we manually calculated the NPS for all scales by subtracting the percentage of detractors (people who scored between 0 and 6) from the percentage of promoters (people who scored 9 or 10).¹³ NPS are widely used in the service industry to assess customer satisfaction, and scores range between –100 and 100.

An a priori power analysis indicated that in order to find a difference in satisfaction of 0.5 on a 0-10 scale, with an effect size of 80% and alpha set at 0.05, we needed 128 patients (64 per group). Since we wanted to compare 4 satisfaction scales, we doubled this.

Results

Patient Characteristics

All patients invited to participate in this study agreed, and 258 patients with various diagnoses were enrolled. The median age of the cohort was 54 years (IQR, 40-65 years); 114 (44%) were men, and 119 (42%) were new patients (Table 1). The number of patients assigned to scales 1, 2, 3, and 4 were 62 (24%), 70 (27%), 67 (26%), and 59 (23%), respectively.

Difference in Distribution

Looking at the data distribution (Figure 2) and skewness and kurtosis (Table 2) of the scales, we found that none of the scales was normally distributed.

Difference in Satisfaction Scores

Mean (SD) scaled satisfaction scores (range, 0-10) were 8.3 (1.2) for the 11-point ordinal scale, 8.3 (1.2) for the 5-point Likert scale, 8.9 (1.7) for the 0-100 numerical VAS, and 8.3 (1.3) for the 0-100 nonnumerical VAS (Table 3 and Table 4).

Difference in Floor and Ceiling Effect

A difference was found in ceiling effect between the different scales (P = 0.025), with the 0-100 numerical VAS showing the highest ceiling effect (34%) and the 0-100 nonnumerical VAS showing the lowest ceiling effect (12%; Table 2). There was no floor effect. A single patient used the lowest score (on the Likert scale).

Correlation Between Satisfaction and Psychological Status

Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064; not in table), indicating that patients with more self-efficacy had higher satisfaction ratings.

Net Promoter Scores

NPS were 35 for the 11-point ordinal scale; 16 for the 5-point Likert scale; 67 for the 0-100 numerical VAS; and 20 for the 0-100 nonnumerical VAS.

Single-question measures of satisfaction can decrease patient burden and limit overlap with measures of communication effectiveness and perceived empathy. Both long and short questionnaires addressing satisfaction and perceived empathy show substantial ceiling effect. We compared 4 different measures for overall scores, floor and ceiling effect, and skewness and kurtosis, and assessed the correlation between scaled satisfaction and psychological status. We found that scale type influenced the median helpfulness score. As one would expect, scales with less ceiling effect have lower median scores. In other words, if the goal is to collect meaningful information and identify areas for improvement, there must be a willingness to accept lower scores.

Only the nonnumerical VAS was below the threshold of 15% ceiling effect proposed by Terwee et al.¹⁴ This scale with 3 anchor points and no visible numbers showed the least ceiling effect (12%) and minimal skew (–1.0), and was closer to kurtosis consistent with a normal distribution (5.0). However, the 11-point ordinal Likert scale with 5 anchor points and visible numbers had the lowest skewness and kurtosis (–0.58 and 4.0). The low ceiling effect observed with the nonnumerical VAS (12%) might be explained by the fact that the scale does not lead patients to a specific description of the helpfulness of their visit, but rather asks patients to use their own judgement in making the rating. The ordinal scale approached the most normal data distribution, and this might be explained by the presence of numbers on the scale. Ratings based on a 0-10 scale are commonly used, and familiarity with the system might have allowed people to pick a number that represents their actual view of the visit helpfulness, rather than picking the highest possible choice (which would have led to a ceiling effect). Study results comparing Likert scales and VAS are conflicting,¹⁵ with some preferring Likert scales for their responsiveness¹⁶ and ease of use in practice,¹⁷ and others preferring VAS for their sensitivity to describe continuous, subjective phenomenon and their high validity and reliability.¹⁸ Looking at our nonnumerical VAS, adding numbers to a scale might not help avoid, and may actually increase, the presence of ceiling effect. However, with the ordinal scale with visible numbers, we saw a 21% ceiling effect coupled with low skew and kurtosis (–0.58 and 4.0), which indicate that the distribution of scores is relatively normal. This finding is in line with other study results.¹⁹

Our findings demonstrated that feedback concerning self-efficacy, health anxiety, or depression had no or only a small effect on patient satisfaction. Consistent with prior evidence, psychological factors had limited or no correlation with satisfaction.^20-24 Given the effect that priming has on patient-reported outcome measures, the effect of psychological factors on satisfaction could be an area of future study.

The NPS varied substantially based on scale structure. Increasing the spread of the scores to limit the ceiling effect will likely reduce promoters and detractors and increase neutrals. NPS systems have been used in the past to measure patient satisfaction with common hand surgery techniques and with community mental health services.^25,26 These studies suggest that NPS could be a helpful addition to commonly used clinical measures of satisfaction, after more research has been done to validate it. The evidence showing that NPS are strongly influenced by scale structure suggests that NPS should be used and interpreted with caution.

Several caveats regarding this study should be kept in mind. This study specifically addressed ratings of visit helpfulness. Differently phrased questions might lead to different results. More work is needed to determine the essence of satisfaction with a medical visit.¹ In addition, the majority of our patient population was white, employed, and privately insured, limiting generalizability to other populations with different demographics. Finally, all patients were seen by an orthopedic surgeon, and our results might not apply to other populations or clinical settings. However, given the scope of this study, we suspect that the findings can be generalized to specialty care in general and likely all medical contexts.

Conclusion

It is clear from this work that scale design can affect ceiling effect. We plan to test alternative phrasings and structures of single-question measures of satisfaction with a medical visit so that we can better study what factors contribute to satisfaction. It is notable that this approach runs counter to efforts to improve satisfaction scores, because reducing the ceiling effect reduces the mean score and may contribute to worse NPS. Further study is needed to find the optimal measure to assess satisfaction ratings.

Corresponding author: David Ring, MD, PhD, 1701 Trinity Street, Austin, TX, 78712; david.ring@austin.utexas.edu.

Financial disclosures: Dr. Ring has or may receive payment or benefits from Skeletal Dynamics; Wright Medical Group; the journal Clinical Orthopaedics and Related Research; and universities, hospitals, and lawyers not related to the submitted work.

From Dell Medical School, The University of Texas at Austin, Austin, TX.

Abstract

• Objective: Satisfaction measures often show substantial ceiling effects. This randomized controlled trial tested the null hypothesis that there is no difference in mean overall satisfaction, ceiling and floor effect, and data distribution between 4 different kinds of single-question scales assessing the helpfulness of a visit. We also hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared with the Net Promoter Scores (NPS).
• Design: Randomized controlled trial.
• Methods: We enrolled 258 adult, English-speaking new and returning patients. Patients were randomly assigned to 1 of 4 different scale types: (1) an 11-point ordinal scale with 5 anchor points; (2) a 5-point Likert scale; (3) a 0-100 visual analogue scale (VAS) electronic slider with 3 anchor points and visible numbers; and (4) a 0-100 VAS with 3 anchor points and no visible numbers. Additionally, patients completed the 2-item Pain Self-Efficacy Questionnaire (PSEQ-2), 5-item Short Health Anxiety Inventory scale (SHAI-5), and Patient-Reported Outcomes Measurement Information System (PROMIS) Depression. We assessed mean and median score, floor and ceiling effect, and skewness and kurtosis for each scale. Spearman correlation tests were used to test correlations between satisfaction and psychological status.
• Results: The nonnumerical 0-100 VAS with 3 anchor points and the 5-point Likert scale had the least ceiling effect (12% and 20%, respectively). The 11-point ordinal scale had skewness and kurtosis closest to a normal distribution (skew = –0.58 and kurtosis = 4.0). Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064). NPS were 35, 16, 67, and 20 for the scales, respectively.
• Conclusion: Single-question measures of satisfaction can be adjusted to limit the ceiling effect. Additional research in this area is warranted.

Keywords: patient satisfaction; floor and ceiling effect; skewness and kurtosis; quality improvement.

Patient satisfaction is an important quality metric that is increasingly being measured, reported, and incentivized. A qualitative study identified 7 themes influencing satisfaction among people visiting an orthopedic surgeon’s office: trust, relatedness, expectations, wait time, visit duration, communication, and empathy.¹ However, another study found that satisfaction and perceived empathy are not associated with wait time or visit duration, but rather with the quality of the visit.² Satisfaction measures that incorporate many of these features in relatively long questionnaires are associated with lower response rates³ and overlap with the factors whose influence on satisfaction one would like to study (eg, perceived empathy or communication effectiveness).⁴ Single- and multiple-question satisfaction scores are prone to a strong right skew, with a substantial ceiling effect.⁵ Ceiling effect occurs when a considerable proportion (about half) of participants select 1 of the top 2 scores (or the maximum score). An ideal scale would measure satisfaction independent from other factors, would use 1 or just a few questions, and would have little or no ceiling effect.

In this randomized controlled trial, we examined whether there were significant differences in mean and median satisfaction, floor and ceiling effect, and data distribution (by looking at skewness and kurtosis) between 4 different kinds of satisfaction scales asking about the helpfulness of a visit. Additionally, we hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared to the Net Promoter Scores (NPS). NPS are commonly used in the service industry to measure customer satisfaction; we are using these scores as a measure of patient satisfaction.

Methods

Study Design

All English-speaking new and return patients ages 18 to 89 years visiting an orthopedic surgeon in 1 of 7 clinics located in a large urban area were considered eligible for this study. Enrollment took place intermittently over a 5-month period. We were granted a waiver of written informed consent. Patients indicated their consent by completing the surveys. Patients were randomly assigned to 1 of the 4 questionnaires containing different scale types using an Excel random-number generator. After the visit, patients were asked to complete the survey. All questionnaires were administered on an encrypted tablet via a HIPAA-compliant, secure web-based application for building and managing online surveys and databases (REDCap; Research Electronic Data Capture).⁶ This study was approved by our Institutional Review Board and is registered on ClinicalTrials.gov (NCT03686735).⁷

Outcome Measures

Study participants were asked to complete questionnaires regarding demographics (sex, age, race/ethnicity, marital status, level of education, work status, insurance status, comorbidities) and to rate satisfaction with their visit on the scale that was randomly assigned to them: (1) an 11-point Likert scale with 5 anchor points and visible numbers; (2) a 5-point Likert scale with 5 anchor points and no visible numbers; (3) a 0-100 VAS with 3 anchor points and visible numbers; (4) a 0-100 VAS with 3 anchor points and no visible numbers (Figure 1). The 4 scales should not differ in time needed to complete them; however, we did not explicitly measure time to completion. Participants also completed measures of psychological aspects of illness. The 2-item Pain Self-Efficacy Questionnaire (PSEQ-2) was used to measure pain self-efficacy, an effective coping strategy for pain.⁸ Higher PSEQ-2 scores indicate a higher level of pain self-efficacy. The 5-item Short Health Anxiety Inventory scale (SHAI-5) was also administered; higher scores on this scale indicate a greater degree of health anxiety.⁹ The Patient-Reported Outcomes Measurement Information System (PROMIS) Depression was used to measure symptoms of depression.¹⁰ Finally, the diagnosis was recorded by the surgeon (not in table).

Statistical Analysis

We reported continuous variables using mean, standard deviation (SD), median, and interquartile range (IQR). Categorical data are presented as frequencies and percentages. We calculated floor and ceiling effect and the skewness and kurtosis of every scale. We scaled every scale to 10 and also standardized every scale. We used the Kruskal–Wallis test to compare differences in satisfaction between the scales; Fisher’s exact test to compare differences in floor and ceiling effect; and Spearman correlation tests to test the correlation between scaled satisfaction scores and psychological status.

Ceiling effects are present when patients select the highest value on a scale rather than a value that reflects their actual feelings about a certain topic. Floor effects are present when patients select the lowest value in a similar fashion. These 2 effects indicate that an independent variable no longer influences the dependent variable being tested. Skewness and kurtosis are rough indicators of a normal distribution of values. Skewness (γ1) is an index of the symmetry of a distribution, with symmetric distributions having a skewness of 0. If skewness has a positive value, it suggests relatively many low values, having a long right tail. Negative skewness suggests relatively many high values, having a long left tail. Kurtosis (γ2) is a measure to describe tailedness of a distribution. Kurtosis of a normal distribution is 3. Negative kurtosis represents little peaked distribution, and positive kurtosis represents more peaked distribution.^11,12 If skewness is 0 and kurtosis is 3, there is a normal, or Gaussian, distribution.

Finally, we manually calculated the NPS for all scales by subtracting the percentage of detractors (people who scored between 0 and 6) from the percentage of promoters (people who scored 9 or 10).¹³ NPS are widely used in the service industry to assess customer satisfaction, and scores range between –100 and 100.

An a priori power analysis indicated that in order to find a difference in satisfaction of 0.5 on a 0-10 scale, with an effect size of 80% and alpha set at 0.05, we needed 128 patients (64 per group). Since we wanted to compare 4 satisfaction scales, we doubled this.

Results

Patient Characteristics

All patients invited to participate in this study agreed, and 258 patients with various diagnoses were enrolled. The median age of the cohort was 54 years (IQR, 40-65 years); 114 (44%) were men, and 119 (42%) were new patients (Table 1). The number of patients assigned to scales 1, 2, 3, and 4 were 62 (24%), 70 (27%), 67 (26%), and 59 (23%), respectively.

Difference in Distribution

Looking at the data distribution (Figure 2) and skewness and kurtosis (Table 2) of the scales, we found that none of the scales was normally distributed.

Difference in Satisfaction Scores

Mean (SD) scaled satisfaction scores (range, 0-10) were 8.3 (1.2) for the 11-point ordinal scale, 8.3 (1.2) for the 5-point Likert scale, 8.9 (1.7) for the 0-100 numerical VAS, and 8.3 (1.3) for the 0-100 nonnumerical VAS (Table 3 and Table 4).

Difference in Floor and Ceiling Effect

A difference was found in ceiling effect between the different scales (P = 0.025), with the 0-100 numerical VAS showing the highest ceiling effect (34%) and the 0-100 nonnumerical VAS showing the lowest ceiling effect (12%; Table 2). There was no floor effect. A single patient used the lowest score (on the Likert scale).

Correlation Between Satisfaction and Psychological Status

Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064; not in table), indicating that patients with more self-efficacy had higher satisfaction ratings.

Net Promoter Scores

NPS were 35 for the 11-point ordinal scale; 16 for the 5-point Likert scale; 67 for the 0-100 numerical VAS; and 20 for the 0-100 nonnumerical VAS.

Single-question measures of satisfaction can decrease patient burden and limit overlap with measures of communication effectiveness and perceived empathy. Both long and short questionnaires addressing satisfaction and perceived empathy show substantial ceiling effect. We compared 4 different measures for overall scores, floor and ceiling effect, and skewness and kurtosis, and assessed the correlation between scaled satisfaction and psychological status. We found that scale type influenced the median helpfulness score. As one would expect, scales with less ceiling effect have lower median scores. In other words, if the goal is to collect meaningful information and identify areas for improvement, there must be a willingness to accept lower scores.

Only the nonnumerical VAS was below the threshold of 15% ceiling effect proposed by Terwee et al.¹⁴ This scale with 3 anchor points and no visible numbers showed the least ceiling effect (12%) and minimal skew (–1.0), and was closer to kurtosis consistent with a normal distribution (5.0). However, the 11-point ordinal Likert scale with 5 anchor points and visible numbers had the lowest skewness and kurtosis (–0.58 and 4.0). The low ceiling effect observed with the nonnumerical VAS (12%) might be explained by the fact that the scale does not lead patients to a specific description of the helpfulness of their visit, but rather asks patients to use their own judgement in making the rating. The ordinal scale approached the most normal data distribution, and this might be explained by the presence of numbers on the scale. Ratings based on a 0-10 scale are commonly used, and familiarity with the system might have allowed people to pick a number that represents their actual view of the visit helpfulness, rather than picking the highest possible choice (which would have led to a ceiling effect). Study results comparing Likert scales and VAS are conflicting,¹⁵ with some preferring Likert scales for their responsiveness¹⁶ and ease of use in practice,¹⁷ and others preferring VAS for their sensitivity to describe continuous, subjective phenomenon and their high validity and reliability.¹⁸ Looking at our nonnumerical VAS, adding numbers to a scale might not help avoid, and may actually increase, the presence of ceiling effect. However, with the ordinal scale with visible numbers, we saw a 21% ceiling effect coupled with low skew and kurtosis (–0.58 and 4.0), which indicate that the distribution of scores is relatively normal. This finding is in line with other study results.¹⁹

Our findings demonstrated that feedback concerning self-efficacy, health anxiety, or depression had no or only a small effect on patient satisfaction. Consistent with prior evidence, psychological factors had limited or no correlation with satisfaction.^20-24 Given the effect that priming has on patient-reported outcome measures, the effect of psychological factors on satisfaction could be an area of future study.

The NPS varied substantially based on scale structure. Increasing the spread of the scores to limit the ceiling effect will likely reduce promoters and detractors and increase neutrals. NPS systems have been used in the past to measure patient satisfaction with common hand surgery techniques and with community mental health services.^25,26 These studies suggest that NPS could be a helpful addition to commonly used clinical measures of satisfaction, after more research has been done to validate it. The evidence showing that NPS are strongly influenced by scale structure suggests that NPS should be used and interpreted with caution.

Several caveats regarding this study should be kept in mind. This study specifically addressed ratings of visit helpfulness. Differently phrased questions might lead to different results. More work is needed to determine the essence of satisfaction with a medical visit.¹ In addition, the majority of our patient population was white, employed, and privately insured, limiting generalizability to other populations with different demographics. Finally, all patients were seen by an orthopedic surgeon, and our results might not apply to other populations or clinical settings. However, given the scope of this study, we suspect that the findings can be generalized to specialty care in general and likely all medical contexts.

Conclusion

It is clear from this work that scale design can affect ceiling effect. We plan to test alternative phrasings and structures of single-question measures of satisfaction with a medical visit so that we can better study what factors contribute to satisfaction. It is notable that this approach runs counter to efforts to improve satisfaction scores, because reducing the ceiling effect reduces the mean score and may contribute to worse NPS. Further study is needed to find the optimal measure to assess satisfaction ratings.

Corresponding author: David Ring, MD, PhD, 1701 Trinity Street, Austin, TX, 78712; david.ring@austin.utexas.edu.

Financial disclosures: Dr. Ring has or may receive payment or benefits from Skeletal Dynamics; Wright Medical Group; the journal Clinical Orthopaedics and Related Research; and universities, hospitals, and lawyers not related to the submitted work.

From Dell Medical School, The University of Texas at Austin, Austin, TX.

Abstract

• Objective: Satisfaction measures often show substantial ceiling effects. This randomized controlled trial tested the null hypothesis that there is no difference in mean overall satisfaction, ceiling and floor effect, and data distribution between 4 different kinds of single-question scales assessing the helpfulness of a visit. We also hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared with the Net Promoter Scores (NPS).
• Design: Randomized controlled trial.
• Methods: We enrolled 258 adult, English-speaking new and returning patients. Patients were randomly assigned to 1 of 4 different scale types: (1) an 11-point ordinal scale with 5 anchor points; (2) a 5-point Likert scale; (3) a 0-100 visual analogue scale (VAS) electronic slider with 3 anchor points and visible numbers; and (4) a 0-100 VAS with 3 anchor points and no visible numbers. Additionally, patients completed the 2-item Pain Self-Efficacy Questionnaire (PSEQ-2), 5-item Short Health Anxiety Inventory scale (SHAI-5), and Patient-Reported Outcomes Measurement Information System (PROMIS) Depression. We assessed mean and median score, floor and ceiling effect, and skewness and kurtosis for each scale. Spearman correlation tests were used to test correlations between satisfaction and psychological status.
• Results: The nonnumerical 0-100 VAS with 3 anchor points and the 5-point Likert scale had the least ceiling effect (12% and 20%, respectively). The 11-point ordinal scale had skewness and kurtosis closest to a normal distribution (skew = –0.58 and kurtosis = 4.0). Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064). NPS were 35, 16, 67, and 20 for the scales, respectively.
• Conclusion: Single-question measures of satisfaction can be adjusted to limit the ceiling effect. Additional research in this area is warranted.

Keywords: patient satisfaction; floor and ceiling effect; skewness and kurtosis; quality improvement.

Patient satisfaction is an important quality metric that is increasingly being measured, reported, and incentivized. A qualitative study identified 7 themes influencing satisfaction among people visiting an orthopedic surgeon’s office: trust, relatedness, expectations, wait time, visit duration, communication, and empathy.¹ However, another study found that satisfaction and perceived empathy are not associated with wait time or visit duration, but rather with the quality of the visit.² Satisfaction measures that incorporate many of these features in relatively long questionnaires are associated with lower response rates³ and overlap with the factors whose influence on satisfaction one would like to study (eg, perceived empathy or communication effectiveness).⁴ Single- and multiple-question satisfaction scores are prone to a strong right skew, with a substantial ceiling effect.⁵ Ceiling effect occurs when a considerable proportion (about half) of participants select 1 of the top 2 scores (or the maximum score). An ideal scale would measure satisfaction independent from other factors, would use 1 or just a few questions, and would have little or no ceiling effect.

In this randomized controlled trial, we examined whether there were significant differences in mean and median satisfaction, floor and ceiling effect, and data distribution (by looking at skewness and kurtosis) between 4 different kinds of satisfaction scales asking about the helpfulness of a visit. Additionally, we hypothesized that there is no correlation between scaled satisfaction and psychological status. Finally, we assessed how the satisfaction scores compared to the Net Promoter Scores (NPS). NPS are commonly used in the service industry to measure customer satisfaction; we are using these scores as a measure of patient satisfaction.

Methods

Study Design

All English-speaking new and return patients ages 18 to 89 years visiting an orthopedic surgeon in 1 of 7 clinics located in a large urban area were considered eligible for this study. Enrollment took place intermittently over a 5-month period. We were granted a waiver of written informed consent. Patients indicated their consent by completing the surveys. Patients were randomly assigned to 1 of the 4 questionnaires containing different scale types using an Excel random-number generator. After the visit, patients were asked to complete the survey. All questionnaires were administered on an encrypted tablet via a HIPAA-compliant, secure web-based application for building and managing online surveys and databases (REDCap; Research Electronic Data Capture).⁶ This study was approved by our Institutional Review Board and is registered on ClinicalTrials.gov (NCT03686735).⁷

Outcome Measures

Study participants were asked to complete questionnaires regarding demographics (sex, age, race/ethnicity, marital status, level of education, work status, insurance status, comorbidities) and to rate satisfaction with their visit on the scale that was randomly assigned to them: (1) an 11-point Likert scale with 5 anchor points and visible numbers; (2) a 5-point Likert scale with 5 anchor points and no visible numbers; (3) a 0-100 VAS with 3 anchor points and visible numbers; (4) a 0-100 VAS with 3 anchor points and no visible numbers (Figure 1). The 4 scales should not differ in time needed to complete them; however, we did not explicitly measure time to completion. Participants also completed measures of psychological aspects of illness. The 2-item Pain Self-Efficacy Questionnaire (PSEQ-2) was used to measure pain self-efficacy, an effective coping strategy for pain.⁸ Higher PSEQ-2 scores indicate a higher level of pain self-efficacy. The 5-item Short Health Anxiety Inventory scale (SHAI-5) was also administered; higher scores on this scale indicate a greater degree of health anxiety.⁹ The Patient-Reported Outcomes Measurement Information System (PROMIS) Depression was used to measure symptoms of depression.¹⁰ Finally, the diagnosis was recorded by the surgeon (not in table).

Statistical Analysis

We reported continuous variables using mean, standard deviation (SD), median, and interquartile range (IQR). Categorical data are presented as frequencies and percentages. We calculated floor and ceiling effect and the skewness and kurtosis of every scale. We scaled every scale to 10 and also standardized every scale. We used the Kruskal–Wallis test to compare differences in satisfaction between the scales; Fisher’s exact test to compare differences in floor and ceiling effect; and Spearman correlation tests to test the correlation between scaled satisfaction scores and psychological status.

Ceiling effects are present when patients select the highest value on a scale rather than a value that reflects their actual feelings about a certain topic. Floor effects are present when patients select the lowest value in a similar fashion. These 2 effects indicate that an independent variable no longer influences the dependent variable being tested. Skewness and kurtosis are rough indicators of a normal distribution of values. Skewness (γ1) is an index of the symmetry of a distribution, with symmetric distributions having a skewness of 0. If skewness has a positive value, it suggests relatively many low values, having a long right tail. Negative skewness suggests relatively many high values, having a long left tail. Kurtosis (γ2) is a measure to describe tailedness of a distribution. Kurtosis of a normal distribution is 3. Negative kurtosis represents little peaked distribution, and positive kurtosis represents more peaked distribution.^11,12 If skewness is 0 and kurtosis is 3, there is a normal, or Gaussian, distribution.

Finally, we manually calculated the NPS for all scales by subtracting the percentage of detractors (people who scored between 0 and 6) from the percentage of promoters (people who scored 9 or 10).¹³ NPS are widely used in the service industry to assess customer satisfaction, and scores range between –100 and 100.

An a priori power analysis indicated that in order to find a difference in satisfaction of 0.5 on a 0-10 scale, with an effect size of 80% and alpha set at 0.05, we needed 128 patients (64 per group). Since we wanted to compare 4 satisfaction scales, we doubled this.

Results

Patient Characteristics

All patients invited to participate in this study agreed, and 258 patients with various diagnoses were enrolled. The median age of the cohort was 54 years (IQR, 40-65 years); 114 (44%) were men, and 119 (42%) were new patients (Table 1). The number of patients assigned to scales 1, 2, 3, and 4 were 62 (24%), 70 (27%), 67 (26%), and 59 (23%), respectively.

Difference in Distribution

Looking at the data distribution (Figure 2) and skewness and kurtosis (Table 2) of the scales, we found that none of the scales was normally distributed.

Difference in Satisfaction Scores

Mean (SD) scaled satisfaction scores (range, 0-10) were 8.3 (1.2) for the 11-point ordinal scale, 8.3 (1.2) for the 5-point Likert scale, 8.9 (1.7) for the 0-100 numerical VAS, and 8.3 (1.3) for the 0-100 nonnumerical VAS (Table 3 and Table 4).

Difference in Floor and Ceiling Effect

A difference was found in ceiling effect between the different scales (P = 0.025), with the 0-100 numerical VAS showing the highest ceiling effect (34%) and the 0-100 nonnumerical VAS showing the lowest ceiling effect (12%; Table 2). There was no floor effect. A single patient used the lowest score (on the Likert scale).

Correlation Between Satisfaction and Psychological Status

Scaled satisfaction scores had a small but significant correlation with PSEQ-2 (r = 0.17; P = 0.006), but not with SHAI-5 (r = –0.12; P = 0.052) or PROMIS Depression (r = –0.12; P = 0.064; not in table), indicating that patients with more self-efficacy had higher satisfaction ratings.

Net Promoter Scores

NPS were 35 for the 11-point ordinal scale; 16 for the 5-point Likert scale; 67 for the 0-100 numerical VAS; and 20 for the 0-100 nonnumerical VAS.

Single-question measures of satisfaction can decrease patient burden and limit overlap with measures of communication effectiveness and perceived empathy. Both long and short questionnaires addressing satisfaction and perceived empathy show substantial ceiling effect. We compared 4 different measures for overall scores, floor and ceiling effect, and skewness and kurtosis, and assessed the correlation between scaled satisfaction and psychological status. We found that scale type influenced the median helpfulness score. As one would expect, scales with less ceiling effect have lower median scores. In other words, if the goal is to collect meaningful information and identify areas for improvement, there must be a willingness to accept lower scores.

Only the nonnumerical VAS was below the threshold of 15% ceiling effect proposed by Terwee et al.¹⁴ This scale with 3 anchor points and no visible numbers showed the least ceiling effect (12%) and minimal skew (–1.0), and was closer to kurtosis consistent with a normal distribution (5.0). However, the 11-point ordinal Likert scale with 5 anchor points and visible numbers had the lowest skewness and kurtosis (–0.58 and 4.0). The low ceiling effect observed with the nonnumerical VAS (12%) might be explained by the fact that the scale does not lead patients to a specific description of the helpfulness of their visit, but rather asks patients to use their own judgement in making the rating. The ordinal scale approached the most normal data distribution, and this might be explained by the presence of numbers on the scale. Ratings based on a 0-10 scale are commonly used, and familiarity with the system might have allowed people to pick a number that represents their actual view of the visit helpfulness, rather than picking the highest possible choice (which would have led to a ceiling effect). Study results comparing Likert scales and VAS are conflicting,¹⁵ with some preferring Likert scales for their responsiveness¹⁶ and ease of use in practice,¹⁷ and others preferring VAS for their sensitivity to describe continuous, subjective phenomenon and their high validity and reliability.¹⁸ Looking at our nonnumerical VAS, adding numbers to a scale might not help avoid, and may actually increase, the presence of ceiling effect. However, with the ordinal scale with visible numbers, we saw a 21% ceiling effect coupled with low skew and kurtosis (–0.58 and 4.0), which indicate that the distribution of scores is relatively normal. This finding is in line with other study results.¹⁹

Our findings demonstrated that feedback concerning self-efficacy, health anxiety, or depression had no or only a small effect on patient satisfaction. Consistent with prior evidence, psychological factors had limited or no correlation with satisfaction.^20-24 Given the effect that priming has on patient-reported outcome measures, the effect of psychological factors on satisfaction could be an area of future study.

The NPS varied substantially based on scale structure. Increasing the spread of the scores to limit the ceiling effect will likely reduce promoters and detractors and increase neutrals. NPS systems have been used in the past to measure patient satisfaction with common hand surgery techniques and with community mental health services.^25,26 These studies suggest that NPS could be a helpful addition to commonly used clinical measures of satisfaction, after more research has been done to validate it. The evidence showing that NPS are strongly influenced by scale structure suggests that NPS should be used and interpreted with caution.

Several caveats regarding this study should be kept in mind. This study specifically addressed ratings of visit helpfulness. Differently phrased questions might lead to different results. More work is needed to determine the essence of satisfaction with a medical visit.¹ In addition, the majority of our patient population was white, employed, and privately insured, limiting generalizability to other populations with different demographics. Finally, all patients were seen by an orthopedic surgeon, and our results might not apply to other populations or clinical settings. However, given the scope of this study, we suspect that the findings can be generalized to specialty care in general and likely all medical contexts.

Conclusion

It is clear from this work that scale design can affect ceiling effect. We plan to test alternative phrasings and structures of single-question measures of satisfaction with a medical visit so that we can better study what factors contribute to satisfaction. It is notable that this approach runs counter to efforts to improve satisfaction scores, because reducing the ceiling effect reduces the mean score and may contribute to worse NPS. Further study is needed to find the optimal measure to assess satisfaction ratings.

Corresponding author: David Ring, MD, PhD, 1701 Trinity Street, Austin, TX, 78712; david.ring@austin.utexas.edu.

Financial disclosures: Dr. Ring has or may receive payment or benefits from Skeletal Dynamics; Wright Medical Group; the journal Clinical Orthopaedics and Related Research; and universities, hospitals, and lawyers not related to the submitted work.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; kfreund@tuftsmedicalcenter.org.

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; kfreund@tuftsmedicalcenter.org.

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From Tufts Medical Center, Boston, MA (Dr. Trautwein and Dr. Schwartz contributed equally to this article).

Abstract

• Objective: To examine whether an institution- and service-specific readmission prediction instrument has improved performance compared to universal tools.
• Design: Retrospective cohort study.
• Setting: Academic medical center located in Boston, MA.
• Participants: Adult inpatients admitted to a medicine service.
• Measurements: Patient attributes, inpatient service assignment, and 30-day readmission rate.
• Results: Of 7972 index admissions, 12.6% were readmitted within 30 days. Thirty-day readmissions were associated with number of medications on admission (adjusted odds ratio [OR], 1.34; 95% confidence interval [CI], 1.11-1.61) for ≥ 11 compared with ≤ 5 medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and discharge (OR, 2.45; 95% CI, 1.97-3.06) to an acute care facility compared to home without services. The subspecialty services with the highest risk of readmission were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.78-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the LACE index.
• Conclusion: A hospital service–specific 30-day readmission prediction tool showed incrementally improved performance over the widely used LACE index.

Keywords: rehospitalization; quality of care; predictive model; hospital medicine.

Readmissions are a costly problem in the United States. The readmission rate among Medicare beneficiaries aged 65 years and older was 17.1 per 100 live discharges in 2016.¹ Although both the relationship between readmissions and quality of care and the use of readmissions as a quality measure are contested,² efforts to reduce potentially avoidable rehospitalizations have received widespread attention and readmissions are a focus of reimbursement reform.^3-5

The LACE index is a widely used model for readmission risk prediction.⁶ Derived from a study of hospitalized patients in Ontario, Canada, LACE uses information about length of stay, acuity, comorbidities, and emergency department utilization to estimate readmission risk. Models such as LACE+, LACE-rt, and HOSPITAL have tried to improve on LACE’s performance, but models with strong discriminative ability are lacking.^7-9 Institution-specific models exist,¹⁰ as do well-organized multicenter studies,⁷ but their generalizability is limited due to population differences or inclusion of data difficult to extract from patients in real-time, such as data gathered through a socioeconomic questionnaire.

Given hospital-specific differences in operational performance and patient population, we sought to develop a statistical model for 30-day readmission prediction and demonstrate the process that could be utilized by other institutions to identify high-risk patients for intensive case management and discharge planning.

Methods

Study Design

We conducted a retrospective cohort study of all admissions to the medicine service at a 415-bed teaching hospital in Boston, MA, from September 1, 2013, through August 31, 2016. Patients are admitted through the emergency department or directly admitted from hospital-based practices or a statewide network of private practices.

Data Collection

Data were abstracted from electronic medical and billing records from the first (index) admission for each patient during the study period. Thirty-day readmission was defined as an unplanned admission in the 30 days following the index discharge date. We excluded patients readmitted after leaving against medical advice and planned readmissions based on information in discharge summaries.

The study team identified candidate risk factors by referencing related published research and with input from a multidisciplinary task force charged with developing strategies to reduce 30-day readmissions. Task force members included attending and resident physicians, pharmacists, nurses, case managers, and administrators. The task force considered factors that could be extracted from the electronic medical record, including demographics, location of care, and clinical measures such as diagnostic codes, as well as data available in nursing, social work, and case management notes. Decisions regarding potential risk factors were reached within the group based on institutional experience, availability, and quality of data within the electronic record for specific variables, as well as published research on the subject, with the goal of selecting variables that could be easily identified before discharge and used to generate a predictive score for use in discharge planning.

Variables

Variables initially considered for inclusion in univariate analyses included demographic characteristics of age, gender, and a combined race/ethnicity variable, delineated as either non-Hispanic white, non-Hispanic black, Asian/Asian Indian, or Hispanic. Those with race listed as other or missing were set to missing. Primary language was categorized as English versus non-English. We included a number of variables related to the severity of the patient’s medical condition during the index admission, including any stay in an intensive care unit (ICU) and number of medications on admission, divided into 3 groups, 0-5, 6-10, and 11 or more. We also included separate indicators for admissions on warfarin or chronic opioids. Charlson comorbidity score as well as heart failure, diabetes, and chronic obstructive pulmonary disease were included as separate variables, since these specific diagnoses have high comorbidity and risk of readmission.

Because the hospital’s medicine service is divided into subspecialty services, we included the admitting service and discharge unit to assess whether certain teams or units were associated with readmission. Discharge disposition was categorized as home with services (ie, physical therapy and visiting nurse), home without services, skilled nursing facility, acute care facility, or other. We included a variable to assess patient frailty and mobility based on the presence of a physical therapy consult. We incorporated social determinants of health, including insurance coverage (private insurance, Medicare, Medicaid, subsidized, or uninsured); per capita income from the patient’s zip code as a proxy for economic status (divided into quartiles for analysis); and substance abuse and alcohol abuse (based on International Classification of Diseases, 10th revision codes). We considered whether the discharge was on a weekday or weekend, and considered distance to the hospital in relation to Boston, either within route 128 (roughly within 15-20 miles of the medical center), within interstate 495 (roughly within 30-40 miles of the medical center), or beyond this. We considered but were unable to incorporate candidate variables that had inconsistent availability in the electronic medical record, such as the Braden score, level of independence with activities of daily living, nursing-determined fall risk, presence of a social work or nutrition consultation, CAGE questionnaire for alcohol abuse, delirium assessment score, the number of adults living in the home, the number of dependents, and marital status.

Analysis

We created a derivation cohort using admission data from September 1, 2013, through November 30, 2015. We used a backward selection process to include variables in the derivation model. Any variable associated with 30-day readmissions with a P value < 0.10 in univariate analyses was considered as a candidate variable. To be retained in the multivariable model, each variable was required to have a significant association with 30-day readmission at the P < 0.05 level. We used beta coefficients to create a numerical score reflective of probability of readmission.

We then created a validation cohort using admissions data between December 1, 2015, and August 31, 2016. We applied the scoring algorithm from the derivation cohort to the validation cohort and compared the discriminative ability of the 2 models using the area under the receiver operating characteristic (ROC) curve. We also compared the area under the ROC curve of our predictive model to the LACE index using the nonparametric approach of DeLong and colleagues.^8,11

Results

The derivation cohort consisted of 7972 index admissions, of which 12.6% were readmitted within 30 days. The patient population was 45% female, 70% white, and 85% English-speaking, with an average age of 61.4 years (standard deviation, 18.1, Table 1). Most patients had either private insurance (43%) or Medicare (41%).

Many patients were medically complex: 21% required ICU care, 29% were taking 11 or more medications on admission, and 52% had a Charlson score of 2 or more. In the previous 6 months, 14% had an emergency department visit and 16% had an admission or overnight observation. The rate of drug or alcohol abuse was 13%. The majority of patients were discharged home without services (54%), while 23% were discharged home with services, 13% were discharged to a skilled nursing facility, and 9% were discharged to an acute care facility.

Factors Associated With 30-Day Readmission

Table 2 displays the univariate and multivariate associations with 30-day readmissions in the derivation cohort. Variables significantly associated with readmission in univariate analysis were Charlson comorbidity score, history of diabetes, ICU utilization, previous emergency department visit, inpatient or observation stay within 6 months, number of medications on admission, use of opioids on admission, a diagnosis of diabetes, type of insurance, physical therapy consultation, admitting service, and discharge disposition. Variables not significant in univariate analysis included warfarin use, alcohol/drug abuse, distance from the hospital, and demographic variables (age, sex, race, language, income by zip code).

Variables associated with readmission in the final multivariate analysis model included a Charlson score of 2 or higher (compared to a score of 0; odds ratio [OR], 1.36; 95% confidence interval [CI], 1.11-1.66); any ICU stay (OR, 1.29; 95% CI, 1.08-1.53); number of medications on admission (OR, 1.34; 95% CI, 1.11-1.61) for 11 or more compared with 5 or fewer medications; prior admission or overnight observation (OR, 1.89; 95% CI, 1.61-2.23); and disposition on discharge to an acute care facility (OR, 2.45; 95% CI, 1.96-3.06), skilled nursing facility (OR, 1.53; 95% CI, 1.23-1.89), or home with services (OR, 1.64; 95% CI, 1.38-1.94) compared with home discharge without services. The hospital service from which the patient was discharged was significantly associated with readmission; the subspecialty services with the highest odds ratios were bone marrow transplant/hematology (OR, 2.46; 95% CI, 1.77-3.40) and oncology (OR, 2.26; 95% CI, 1.67-3.05), as compared with general medicine/geriatrics.

Model Derivation and Validation

We utilized the beta coefficients from the multivariate analysis to create a scoring tool to predict the likelihood of 30-day readmission. We rounded each beta coefficient and calculated a readmission score by adding together the rounded beta coefficients of each of the significant variables. Table 3 presents the cumulative percentage of discharges at each score level, as well as the calculated cumulative percentage of potential readmissions. For example, in our population, a score of 6 or greater accounted for 18% of all discharges, but 36% of all 30-day readmissions.

The ROC curves for the derivation model and LACE index are shown in the Figure. The C statistic for the derivation cohort was 0.67, as compared with a C statistic of 0.63 for the calculated LACE index (P < 0.0001). The validation cohort had a C statistic similar to that of the derivation cohort (0.66).

Discussion

We developed a predictive model that can be used during admission to stratify patients for intensive case management and discharge planning. The model included Charlson score, ICU utilization, admission to inpatient services or observation, visits to the ED in the past 6 months, number of medications on admission, hospital service, and discharge disposition. The C statistic of 0.67 is better than that of the LACE predictive model for our population, although both reflect only modest predictive value.

While our model, which was developed and validated at a single institution, may not be generalizable to other institutions, the method of developing a readmission risk prediction model specific to an institution is readily replicable. While standardized tools for predicting readmission risk exist, they do not necessarily account for unique patient populations and medical complexity at individual institutions. We examined patients discharged over 3 years from the medicine services, creating a service-specific model. Our approach could lead to the widespread development of service- and institution-specific models that take into account the risks and resources appropriate to each patient population and setting.

Many of the factors included in our model were indicative of the patients’ medical complexity, with Charlson comorbidity score and the number of discharge medications strongly associated with readmission. In the derivation of the LACE model, many patients were middle-aged and independent in their activities of daily living, and more than 75% had a Charlson comorbidity score of 0.6.⁶ In our population, by contrast, the majority of patients had a Charlson score of 2 or greater. Health care utilization was a strong positive predictor in both our model and in LACE. The finding that the Charlson comorbidity score was a better predictor than any single chronic illness suggests that medical complexity and comorbidities increase the likelihood of admission more than any 1 chronic condition. Our model incorporates discharge disposition in readmission prediction, another factor associated with medical complexity and frailty.

A surprising finding in our study was the lack of association between social determinants of health, such as alcohol and drug abuse, and readmission risk. We posit several reasons that may account for this finding. First, the population served by the medical center may be too small or homogeneous with respect to social determinants of health to detect a difference in readmission risk. Second, markers available in the electronic medical record to determine social needs may be too crude to distinguish degrees of vulnerability that increase the risk of readmission. We do not discount the importance of social determinants of health as predictors of readmission risk, but we do acknowledge the limitations of the data incorporated in our model.

Predictive models are useful only if they can be incorporated into workflow to identify high-risk patients. Prior to developing and using our model, we used LACE inconsistently because it required length of stay as 1 of the variables. Because the variables in our model are collected and recorded routinely at admission in our electronic medical record, the readmission risk score is calculated and displayed in a daily high-risk patient report. This automated process has afforded a more consistent and reliable approach to readmission risk assessment than previous efforts to assess the LACE index. Case managers use the high-risk patient report to identify patients who require enhanced care coordination and discharge planning. Since the introduction of this predictive model, we have noted a 10% reduction in the hospital’s 30-day readmission rate.

This project was subject to several limitations. Because data on admissions to other facilities were unavailable, we may have underestimated the risk of readmission to other facilities. Our results may not be generalizable to other organizations, although we believe that the methods are readily replicable. The performance of the model and its replication with a validation cohort are strengths of the approach.

Conclusion

We created a hospital service–specific 30-day readmission prediction tool whose performance improved incrementally over the widely used LACE index. This research suggests that readmission prediction is highly context-specific and that organizations would do well to examine the readmission risk factors most pertinent to the populations they serve. We believe that “customized” readmission risk prediction models for particular services in specific hospitals may offer a superior method to identify high-risk patients who may benefit from individualized care planning. Future research is needed to understand how best to capture information about the attributes of vulnerable populations, so that this information can be incorporated into future risk models.

Corresponding author: Karen Freund, MD, Tufts Medical Center, 800 Washington St., Boston, MA 02111; kfreund@tuftsmedicalcenter.org.

Financial disclosures: None.

Funding for this work was provided by the Commonwealth of Massachusetts, Executive Office of Health and Human Services.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; ddokken@ipfcc.org.

Funding disclosures: None.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; ddokken@ipfcc.org.

Funding disclosures: None.

From the Institute for Patient- and Family-Centered Care, Bethesda, MD (Ms. Dokken and Ms. Johnson), and Northwell Health, New Hyde Park, NY (Dr. Barden, Ms. Tuomey, and Ms. Giammarinaro).

Abstract

Objective: To describe the growth of Better Together: Partnering with Families, a campaign launched in 2014 to eliminate restrictive hospital visiting policies and to put in place policies that recognize families as partners in care, and to discuss the processes involved in implementing the initiative in a large, integrated health system.

Methods: Descriptive report.

Results: In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the Better Together campaign to emphasize the importance of family presence and participation to the quality, experience, safety, and outcomes of care. Since then, this initiative has expanded in both the United States and Canada. With support from 2 funders in the United States, special attention was focused on acute care hospitals across New York State. Nearly 50 hospitals participated in 2 separate but related projects. Fifteen of the hospitals are part of Northwell Health, New York State’s largest health system. Over a 10-month period, these hospitals made significant progress in changing policy, practice, and communication to support family presence.

Conclusion: The Better Together initiative was implemented across a health system with strong support from leadership and the involvement of patient and family advisors. An intervention offering structured training, coaching, and resources, like IPFCC’s Better Together initiative, can facilitate the change process.

Keywords: family presence; visiting policies; patient-centered care; family-centered care; patient experience.

The presence of families at the bedside of patients is often restricted by hospital visiting hours. Hospitals that maintain these restrictive policies cite concerns about negative impacts on security, infection control, privacy, and staff workload. But there are no data to support these concerns, and the experience of hospitals that have successfully changed policy and practice to welcome families demonstrates the potential positive impacts of less restrictive policies on patient care and outcomes.¹ For example, hospitalization can lead to reduced cognitive function in elderly patients. Family members would recognize the changes and could provide valuable information to hospital staff, potentially improving outcomes.²

In June 2014, the Institute for Patient- and Family-Centered Care (IPFCC) launched the campaign Better Together: Partnering with Families.³ The campaign is is grounded in patient- and family- centered care, an approach to care that supports partnerships among health care providers, patients, and families, and, among other core principles, advocates that patients define their “families” and how they will participate in care and decision-making.

Emphasizing the importance of family presence and participation to quality and safety, the Better Together campaign seeks to eliminate restrictive visiting policies and calls upon hospitals to include families as members of the care team and to welcome them 24 hours a day, 7 days a week, according to patient preference. As part of the campaign, IPFCC developed an extensive toolkit of resources that is available to hospitals and other organizations at no cost. The resources include sample policies; profiles of hospitals that have implemented family presence policies; educational materials for staff, patients, and families; and a template for hospital websites. This article, a follow-up to an article published in the January 2015 issue of JCOM,¹ discusses the growth of the Better Together initiative as well as the processes involved in implementing the initiative across a large health system.

Growth of the Initiative

Since its launch in 2014, the Better Together initiative has continued to expand in the United States and Canada. In Canada, under the leadership of the Canadian Foundation for Healthcare Improvement (CFHI), more than 50 organizations have made a commitment to the Better Together program and family presence.⁴ Utilizing and adapting IPFCC’s Toolkit, CFHI developed a change package of free resources for Canadian organizations.⁵ Some of the materials, including the Pocket Guide for Families (Manuel des Familles), were translated into French.⁶

With support from 2 funders in the United States, the United Hospital Fund and the New York State Health (NYSHealth) Foundation, through a subcontract with the New York Public Interest Research Group (NYPIRG), IPFCC has been able to focus on hospitals in New York City, including public hospitals, and, more broadly, acute care hospitals across New York State. Nearly 50 hospitals participated in these 2 separate but related projects.

Education and Support for New York City Hospitals

Supported by the United Hospital Fund, an 18-month project that focused specifically on New York City hospitals was completed in June 2017. The project began with a 1-day intensive training event with representatives of 21 hospitals. Eighteen of those hospitals were eligible to participate in follow-up consultation provided by IPFCC, and 14 participated in some kind of follow-up. NYC Health + Hospitals (H+H), the system of public hospitals in NYC, participated most fully in these activities.

The outcomes of the Better Together initiative in New York City are summarized in the report Sick, Scared, & Separated From Loved Ones,² which is based on a pre/post review of hospital visitation/family presence policies and website communications. According to the report, hospitals that participated in the IPFCC training and consultation program performed better, as a group, with respect to improved policy and website scores on post review than those that did not. Of the 10 hospitals whose scores improved during the review period, 8 had participated in the IPFCC training and 1 hospital was part of a hospital network that did so. (Six of these hospitals are part of the H+H public hospital system.) Those 9 hospitals saw an average increase in scores of 4.9 points (out of a possible 11). All of the website communication improvements were related to the designation or role of the family member/care partner, or the patient’s right to choose visitors and family members/care partners, fundamental elements of the Better Together initiative.²

A Learning Community for Hospitals in New York State

With support from the NYSHealth Foundation, IPFCC again collaborated with NYPIRG and New Yorkers for Patient & Family Empowerment on a 2-year initiative, completed in November 2019, that involved 26 hospitals: 15 from Northwell Health, New York State’s largest health system, and 11 hospitals from health systems throughout the state (Greater Hudson Valley Health System, now Garnet Health; Mohawk Valley Health System; Rochester Regional Health; and University of Vermont Health Network). An update of the report Sick, Scared, & Separated From Loved Onescompared pre/post reviews of policies and website communications regarding hospital visitation/family presence.⁷ Its findings confirm that hospitals that participated in the Better Together Learning Community improved both their policy and website scores to a greater degree than hospitals that did not participate and that a planned intervention can help facilitate change.

During the survey period, 28 out of 40 hospitals’ website navigability scores improved. Of those, hospitals that did not participate in the Better Together Learning Community saw an average increase in scores of 1.2 points, out of a possible 11, while the participating hospitals saw an average increase of 2.7 points, with the top 5 largest increases in scores belonging to hospitals that participated in the Better Together Learning Community.⁷

Northwell Health is a large integrated health care organization comprising more than 69,000 employees, 23 hospitals, and more than 750 medical practices, located geographically across New York State. Embracing patient- and family-centered care, Northwell is dedicated to improving the quality, experience, and safety of care for patients and their families. Welcoming and including patients, families, and care partners as members of the health care team has always been a core element of Northwell’s organizational goal of providing world-class patient care and experience.

Four years ago, the organization reorganized and formalized a system-wide Patient & Family Partnership Council (PFPC).⁸ Representatives on the PFPC include a Northwell patient experience leader and patient/family co-chair from local councils that have been established in nearly all 23 hospitals as well as service lines. Modeling partnership, the PFPC is grounded in listening to the “voice” of patients and families and promoting collaboration, with the goal of driving change across varied aspects and experiences of health care delivery.

Through the Office of Patient and Customer Experience (OPCE), a partnership with IPFCC and the Better Together Learning Community for Hospitals in New York State was initiated as a fundamental next step in Northwell’s journey to enhance system-wide family presence and participation. Results from Better Together’s Organizational Self-Assessment Tool and process identified opportunities to influence 3 distinct areas: policy/staff education, position descriptions/performance management, and website/signage. Over a 10-month period (September 2018 through June 2019), 15 Northwell hospitals implemened significant patient- and family-centered improvements through multifaceted shared work teams (SWT) that partnered around the common goal of supporting the patient and family experience (Figure). Northwell’s SWT structure allowed teams to meet individually on specific tasks, led by a dedicated staff member of the OPCE to ensure progress, support, and accountability. Six monthly coaching calls or report-out meetings were attended by participating teams, where feedback and recommendations shared by IPFCC were discussed in order to maintain momentum and results.

Policy/Staff Education

The policy/staff education SWT focused on appraising and updating existing policies to ensure alignment with key patient- and family-centered concepts and Better Together principles (Table 1). By establishing representation on the System Policy and Procedure Committee, OPCE enabled patients and families to have a voice at the decision-making table. OPCE leaders presented the ideology and scope of the transformation to this committee. After reviewing all system-wide policies, 4 were identified as key opportunities for revision. One overarching policy titled “Visitation Guidelines” was reviewed and updated to reflect Northwell’s mission of patient- and family-centered care, retiring the reference to “families” as “visitors” in definitions, incorporating language of inclusion and partnership, and citing other related policies. The policy was vetted through a multilayer process of review and stakeholder feedback and was ultimately approved at a system Performance Improvement Coordinating Group meeting under a new title, “Visitation: Presence and Participation of Families, Support System Designees and Visitor(s) in Care.”

Three additional related policies were also updated to reflect core principles of inclusion and partnership. These included system policies focused on discharge planning; identification of health care proxy, agent, support person and caregiver; and standards of behavior not conducive in a health care setting. As a result of this work, OPCE was invited to remain an active member of the System Policy and Procedure Committee, adding meaningful new perspectives to the clinical and administrative policy management process. Once policies were updated and approved, the SWT focused on educating leaders and teams. Using a diversified strategy, education was provided through various modes, including weekly system-wide internal communication channels, patient experience huddle messages, yearly mandatory topics training, and the incorporation of essential concepts in existing educational courses (classroom and e-learning modalities).

Position Descriptions/Performance Management

The position descriptions/performance management SWT focused its efforts on incorporating patient- and family-centered concepts and language into position descriptions and the performance appraisal process (Table 2). Due to the complex nature of this work, the process required collaboration from key subject matter experts in human resources, talent management, corporate compensation, and labor management. In 2019, Northwell began an initiative focused on streamlining and standardizing job titles, roles, and developmental pathways across the system. The overarching goal was to create system-wide consistency and standardization. The SWT was successful in advising the leaders overseeing this job architecture initiative on the importance of including language of patient- and family-centered care, like partnership and collaboration, and of highlighting the critical role of family members as part of the care team in subsequent documents.

Northwell has 6 behavioral expectations, standards to which all team members are held accountable: Patient/Customer Focus, Teamwork, Execution, Organizational Awareness, Enable Change, and Develop Self. As a result of the SWT’s work, Patient/Customer Focus was revised to include “families” as essential care partners, demonstrating Northwell’s ongoing commitment to honoring the role of families as members of the care team. It also ensures that all employees are aligned around this priority, as these expectations are utilized to support areas such as recognition and performance. Collaborating with talent management and organizational development, the SWT reviewed yearly performance management and new-hire evaluations. In doing so, they identified an opportunity to refresh the anchored qualitative rating scales to include behavioral demonstrations of patient- and family-centered care, collaboration, respect, and partnership with family members.

Website/Signage

Websites make an important first impression on patients and families looking for information to best prepare for a hospital experience. Therefore, the website/signage SWT worked to redesign hospital websites, enhance digital signage, and perform a baseline assessment of physical signage across facilities. Initial feedback on Northwell’s websites identified opportunities to include more patient- and family-centered, care-partner-infused language; improve navigation; and streamline click levels for easier access. Content for the websites was carefully crafted in collaboration with Northwell’s internal web team, utilizing IPFCC’s best practice standards as a framework and guide.

Next, a multidisciplinary website shared-governance team was established by the OPCE to ensure that key stakeholders were represented and had the opportunity to review and make recommendations for appropriate language and messaging about family presence and participation. This 13-person team was comprised of patient/family partners, patient-experience culture leaders, quality, compliance, human resources, policy, a chief nursing officer, a medical director, and representation from the Institute for Nursing. After careful review and consideration from Northwell’s family partners and teams, all participating hospital websites were enhanced as of June 2019 to include prominent 1-click access from homepages to information for “patients, families and visitors,” as well as “your care partners” information on the important role of families and care partners.

Along with refreshing websites, another step in Northwell’s work to strengthen messaging about family presence and participation was to partner and collaborate with the system’s digital web team as well as local facility councils to understand the capacity to adjust digital signage across facilities. Opportunities were found to make simple yet effective enhancements to the language and imagery of digital signage upon entry, creating a warmer and more welcoming first impression for patients and families. With patient and family partner feedback, the team designed digital signage with inclusive messaging and images that would circulate appropriately based on the facility. Signage specifically welcomes families and refers to them as members of patients’ care teams.

Northwell’s website/signage SWT also directed a 2-phase physical signage assessment to determine ongoing opportunities to alter signs in areas that particularly impact patients and families, such as emergency departments, main lobbies, cafeterias, surgical waiting areas, and intensive care units. Each hospital’s local PFPC did a “walk-about”⁹ to make enhancements to physical signage, such as removing paper and overcrowded signs, adjusting negative language, ensuring alignment with brand guidelines, and including language that welcomed families. As a result of the team’s efforts around signage, collaboration began with the health system’s signage committee to help standardize signage terminology to reflect family inclusiveness, and to implement the recommendation for a standardized signage shared-governance team to ensure accountability and a patient- and family-centered structure.

Sustainment

Since implementing Better Together, Northwell has been able to infuse a more patient- and family-centered emphasis into its overall patient experience message of “Every role, every person, every moment matters.” As a strategic tool aimed at encouraging leaders, clinicians, and staff to pause and reflect about the “heart” of their work, patient and family stories are now included at the beginning of meetings, forums, and team huddles. Elements of the initiative have been integrated in current Patient and Family Partnership sustainment plans at participating hospitals. Some highlights include continued integration of patient/family partners on committees and councils that impact areas such as way finding, signage, recruitment, new-hire orientation, and community outreach; focus on enhancing partner retention and development programs; and inclusion of patient- and family-centered care and Better Together principles in ongoing leadership meetings.

Factors Contributing to Success

Health care is a complex, regulated, and often bureaucratic world that can be very difficult for patients and families to navigate. The system’s partnership with the Better Together Learning Community for Hospitals in New York State enhanced its efforts to improve family presence and participation and created powerful synergy. The success of this partnership was based on a number of important factors:

A solid foundation of support, structure, and accountability. The OPCE initiated the IPFCC Better Together partnership and established a synergistic collaboration inclusive of leadership, frontline teams, multiple departments, and patient and family partners. As a major strategic component of Northwell’s mission to deliver high-quality, patient- and family-centered care, OPCE was instrumental in connecting key areas and stakeholders and mobilizing the recommendations coming from patients and families.

A visible commitment of leadership at all levels. Partnering with leadership across Northwell’s system required a delineated vision, clear purpose and ownership, and comprehensive implementation and sustainment strategies. The existing format of Northwell’s PFPC provided the structure and framework needed for engaged patient and family input; the OPCE motivated and organized key areas of involvement and led communication efforts across the organization. The IPFCC coaching calls provided the underlying guidance and accountability needed to sustain momentum. As leadership and frontline teams became aware of the vision, they understood the larger connection to the system’s purpose, which ultimately created a clear path for positive change.

Meaningful involvement and input of patient and family partners. Throughout this project, Northwell’s patient/family partners were involved through the PFPC and local councils. For example, patient/family partners attended every IPFCC coaching call; members had a central voice in every decision made within each SWT; and local PFPCs actively participated in physical signage “walk-abouts” across facilities, making key recommendations for improvement. This multifaceted, supportive collaboration created a rejuvenated and purposeful focus for all council members involved. Some of their reactions include, “…I am so happy to be able to help other families in crisis, so that they don’t have to be alone, like I was,” and “I feel how important the patient and family’s voice is … it’s truly a partnership between patients, families, and staff.”

Regular access to IPFCC as a best practice coach and expert resource. Throughout the 10-month process, IPFCC’s Better Together Learning Community for Hospitals in New York State provided ongoing learning interventions for members of the SWT; multiple and varied resources from the Better Together toolkit for adaptation; and opportunities to share and reinforce new, learned expertise with colleagues within the Northwell Health system and beyond through IPFCC’s free online learning community, PFCC.Connect.

Conclusion

Family presence and participation are important to the quality, experience, safety, and outcomes of care. IPFCC’s campaign, Better Together: Partnering with Families, encourages hospitals to change restrictive visiting policies and, instead, to welcome families and caregivers 24 hours a day.

Two projects within Better Together involving almost 50 acute care hospitals in New York State confirm that change in policy, practice, and communication is particularly effective when implemented with strong support from leadership. An intervention like the Better Together Learning Community, offering structured training, coaching, and resources, can facilitate the change process.

Corresponding author: IPFCC, Deborah L. Dokken, 6917 Arlington Rd., Ste. 309, Bethesda, MD 20814; ddokken@ipfcc.org.

Funding disclosures: None.

From West Virginia University, Morgantown, WV.

Abstract

Objective: To describe the strategies a family medicine clinic in Appalachia utilized to increase nephropathy screening rates as well as to explore the factors predictive of nephropathy screening in patients with diabetes.

Design: This quality improvement project targeted the points in the care process when patients are lost to follow-up for nephropathy screening.

Setting and participants: Patients with diabetes cared for by a primary care provider (PCP) at an academic family medicine practice in Appalachia from January 2018 to November 2018.

Interventions: Bulk orders for albumin-to-creatinine (ACR) testing and urine collection during clinic visit, enhanced patient communication through bulk communication reminders and individual patient outreach, and education of clinic providers.

Measurements: Demographic data and monthly nephropathy screening rates.

Results: The nephropathy screening rate increased by 6.2% during the project. Older patients living closer to the clinic who visited their PCP 3 or more times per year were the most likely to be screened.

Conclusion: Combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy treatment and screening in rural patients with diabetes at a family medicine clinic.

Keywords: rural; kidney disease; albumin-to-creatinine ratio; electronic health record.

According to the Centers for Disease Control and Prevention (CDC), an estimated 30.3 million people in the United States—about 9.4% of the population—have been diagnosed with diabetes.¹ Diabetes is the seventh leading cause of death in the United States, and it contributes to other leading causes of death: heart disease and stroke.¹ Diabetes also is related to high morbidity risk and is a leading cause of chronic kidney disease.¹ The total cost of diagnosed diabetes was estimated at $327 billion in direct medical costs and reduced productivity.²

Residents of Appalachia bear a disproportionate burden of diabetes and other related negative health outcomes; these outcomes are influenced by a number of factors, including socioeconomic status, poverty, rurality, and health care access. Rates of chronic disease, such as diabetes, are most pronounced in Appalachia’s most economically distressed counties.^3-5 In 2011, the CDC labeled a 644-county area the “diabetes belt,” which included most of Appalachia.⁶ As a result of this elevated prevalence of diabetes in Appalachia as compared to the rest of the country, complications directly associated with diabetes are more commonly observed in Appalachian residents. One of the most damaging complications is diabetic nephropathy.

Diabetic nephropathy results from damage to the microvasculature of the kidney due to inadequately controlled blood glucose. This, in turn, leads to decreased renal function, eventually leading to clinically significant renal disease. The long-term complications associated with nephropathy can include many comorbid conditions, the most serious of which are progression to end-stage renal disease, dialysis requirement, and early mortality. Diabetic nephropathy affects approximately 40% of patients with type 1 and type 2 diabetes.^7,8

One way to prevent complications of diabetic nephropathy, in addition to good glycemic control in patients with diabetes, is early and regular screening. Currently, the American Diabetes Association (ADA) recommends yearly screening for diabetic nephropathy in the form of a urine albumin-to-creatinine ratio (ACR) for patients 18 to 75 years of age.² This screening to detect diabetic nephropathy is recognized as a marker of quality care by many public and private insurance agencies and medical specialty associations, such as the Centers for Medicare and Medicaid Services.

Many patients with diabetes are cared for by primary care providers (PCP), and these PCP appointments provide an opportune time to screen and appropriately treat nephropathy. Screening opportunities are often missed, however, due to time constraints and competing health priorities. There are also a number of other factors specific to the Appalachian region that reduce the likelihood of screening for diabetic nephropathy, such as a lack of health insurance, the need to travel long distances to see a PCP, work and household responsibilities, low levels of education and health literacy, and a mistrust of outsiders regarding personal matters, including health.^9-11 While nephropathy can have a detrimental impact on patients across populations, it is of particular concern for a state located in the heart of Appalachia, such as West Virginia.

Given the disproportionate burden of diabetes in this region and the potentially severe consequences of undetected nephropathy, clinicians from an academic family medicine clinic in West Virginia undertook a quality improvement project to increase the rate of nephropathy screening and treatment among patients with diabetes. This article describes the intervention strategies the team utilized to increase nephropathy screening and treatment in patients 18 to 75 years of age who met quality measures for nephropathy screening or treatment in the previous 12 months and explores the factors most predictive of nephropathy screening in Appalachian patients in this age group. It also reports the challenges and opportunities encountered and offers suggestions for other providers and clinics attempting to increase their nephropathy screening rates.

Methods

Setting and Study Population

The study population included patients ages 18 to 75 years under the care of providers in an academic family medicine practice in West Virginia who had been diagnosed with diabetes mellitus. The study focused on those patients overdue for diabetic nephropathy screening (ie, had not been screened in previous 12 months). The project began in January 2018 with a screening rate of 83.8%. The goal of this project was to increase this compliance metric by at least 5%. The project protocol was submitted to the West Virginia University Institutional Review Board, and, because it is a quality improvement project, permission was given to proceed without a board review.

Interventions

The team identified and implemented several interventions intended to reduce screening barriers and increase the screening rate.

Bulk orders for ACR and urine collection during clinic visits. Prior to initiation of this project, it was left to individual clinic providers to order nephropathy screening for patients with diabetes during a clinic visit; after receiving the order for “random urine microalbumin/creatinine ratio,” patients then had to travel to a lab to provide a urine sample. For this project and moving forward, the team changed to the procedure of initiating bulk ACR orders and collecting urine samples during clinic visits from all patients ages 18 to 75 years who have diabetes.

Bulk communication reminders. Since many patients with diabetes may not have realized they were overdue for nephropathy screening, the team began sending out bulk communication reminders through either the institution’s electronic health record (EHR; MyChart) or postal service–delivered physical letters (according to patient communication preferences) to remind patients that they were due for screening and to encourage them to schedule an appointment or keep a previously scheduled appointment with their PCP.

Individual patient outreach. A team of pharmacy students led by a licensed pharmacist in the family medicine clinic contacted patients overdue for screening even after bulk communication reminders went out. The students telephoned patients 2 to 3 months following the bulk communication. The students obtained an updated list of patients with diabetes ages 18 to 75 years from an EHR quality report. They began by prescreening the patients on the overdue list for potential candidacy for an angiotensin-converting enzyme (ACE) inhibitor or an angiotensin II receptor blocker (ARB). Screening for candidacy included evaluation of recent blood pressure readings, electrolytes (ie, basic metabolic panel), and ACR. If the students determined a patient was a candidate, they presented the patient to the preceptor for verification and then reached out to the provider with a recommendation. If the provider agreed, the student contacted the patient by telephone for medication counseling and education. The remaining patients determined not to be candidates for ACE inhibitors or ARBs were contacted by the pharmacy students by telephone to remind them that laboratory work was pending. Up to 3 phone call attempts were made before patients were determined to be unreachable. Students left voice mails with generic reminders if a patient could not be reached. If a patient answered, the student provided a reminder but also reviewed indications for lab work, the reason why the provider wished for follow-up, and updated lab hours. Students also followed up with the results of the work-up, as appropriate. During this outreach process, the student team encountered a number of patients who had moved or changed to a PCP outside of the family medicine clinic. In these cases, the EHR was updated and those patients were removed from the list of patients altogether.

Education of clinic providers. Clinic providers were educated during faculty and resident meetings and didactic learning sessions on identifying patients within the EHR who are due for nephropathy screening. They also received instruction on how to update the EHR to reflect completed screenings.

Data Analysis

All analyses in this study were conducted using SAS (version 9.4, 2013, SAS Institute Inc., Cary, NC). Descriptive analyses were conducted to summarize basic patient demographic information. To compare patients screened within the previous 12 months to those patients overdue for screening, 2-sample t-tests were used to examine differences in patients’ age, HbA1c, ACR, and creatinine level and the distance (in miles) between the patient’s home and the clinic. Chi-square analyses were used to examine the relationship between whether a patient was recently screened for nephropathy and the patient’s insurance, number of patient visits in the previous 12 months, and provider level. Logistic regression analyses were conducted to control for covariates and to explore which factors were most predictive of nephropathy screening. All tests were 2-tailed, and P values less than 0.05 were considered statistically significant.

Patient Characteristics

There were 1676 family medicine clinic patients with diabetes between 18 and 75 years of age (Table 1 and Table 2). Of the total sample, 1489 (88.8%) had completed screening for nephropathy in the 12 months prior to evaluation, and 67.5%, 23.7%, and 8.8% of patients had private insurance, Medicare, and Medicaid, respectively.

The mean (SD) age of the patients was 56.3 (11.9) years. The mean distance between the patient’s home and the clinic was 26.6 (76.8) miles. The mean number of visits was 3.6 (2.9) per year, and 43.0% of the patientvisited the clinic more than 3 times in a year. The mean values for HbA1c (%), creatinine (g/mol), and ACR (mg/g) were 7.7 (1.9), 1.0 (0.7), and 9.4 (31.4), respectively.

Screening of Patients for Nephropathy

Patients with Medicare and private insurance were more likely to have completed the nephropathy screening than those with Medicaid (92.5% versus 88.8% versus 82.8%, P = 0.004; Table 3 and Table 4).

Changes in Screening Rate

The practice-wide screening rate was 83.8% at the start of this project in January 2018. The screening rate steadily increased throughout 2018, reaching 90.3% in August 2018, and then leveled off around 90% when the project was concluded at the end of November 2018 (Figure). As an added benefit of the increased screening rates, a number of patients were initiated on an ACE inhibitor or ARB based on the team’s screening efforts.

Predictors of Nephropathy Screening

A logistic regression analysis was conducted with nephropathy screening (screened or not screened) as the outcome and 7 patient characteristics as predictors: type of insurance (private, Medicare, or Medicaid), PCP visits in the past 12 months (≤ 3 or > 3), distance in miles of the patient’s residence from the clinic, age, last HbA1c value, last ACR value, and last creatinine value. A test of the full model with all 7 predictors was statistically significant (χ² (8) = 57.77, P < 0.001). Table 5 shows regression coefficients, Wald statistics, and 95% confidence intervals for odds ratios for each of the 7 predictors. According to the Wald criterion, 3 patient characteristics were significant predictors of nephropathy screening: age, distance between the patient’s home and clinic, and number of PCP visits in the past 12 months. After adjusting for the covariates, there were still significant associations between the nephropathy screening status and age ( χ²(1) = 9.64, P < 0.01); distance between the patient’s home and the clinic (χ²(1) = 3.98, P < 0.05); and the number of PCP visits in the previous year (χ²(1) = 21.74, P < 0.001). With each 1-year increment in age, the odds of completing the nephropathy screening increased by 3.2%. With each 1-mile increase in the distance between the patient’s home and clinic, the odds of completing the nephropathy screening decreased by 0.2%. Patients who visited the clinic more than 3 times in a year were 3.9 times (95% confidence interval, 2.2-7.0) more likely to complete the nephropathy screening than their counterparts who visited fewer than 3 times per year.

In summary, older patients living within about 164 miles of the clinic (ie, within 1 standard deviation from the average miles between patient’s homes and the clinic) who visited their PCP 3 or more times per year were the most likely to be screened.

Diabetic nephropathy is a critical issue facing family medicine providers and patients. The morbidity and mortality costs are significant, as diabetic nephropathy is the leading cause of end-stage renal disease. While the ADA recommends annual ACR screening in patients with diabetes and prescription of ACE inhibitors or ARBs in patients who qualify, many patients do not receive these interventions, despite following up with a provider.^12-15 There is no current literature that indicates the compliance rates in the rural setting. Due to health disparities in the rural setting noted in the literature, it could be hypothesized that these individuals are at high risk of not meeting these screening and treatment recommendations.^16,17 Limited access to care and resources, gaps in insurance coverage, and lower health literacy are a few barriers identified in the rural population that may influence whether these measures are met.¹⁷ 

Considering the disease burden of diabetes and its related complications, including nephropathy, consistent screening is necessary to reduce diabetes-related burdens and cost, while also increasing the quality of life for patients with diabetes. All parties must be involved to ensure appropriate compliance and treatment. Our institution’s implementation of quality improvement strategies has key implications for nephropathy screening and treatment efforts in rural settings.

An additional step of having a health care provider (other than the PCP) screen all patients who are not meeting the standard allows for identification of gaps in care. In our quality improvement workflow, the clinical pharmacist screened all patients for candidacy for ACE inhibitor/ARB therapy. While only a small percentage of patients qualified, many of these patients had previously been on therapy and were discontinued for an unknown reason or were stopped due to an acute condition (eg, acute kidney injury) and never restarted after recovery. Other patients required additional education that therapy would be utilized for nephroprotection versus blood pressure management (secondary to an elevated ACR). This highlights the importance of transitions of care and ongoing, intensive education, not only during initial diagnosis but also throughout the disease-state progression.

Utilization of EHRs and telephone outreach are additional aspects of care that can be provided. Our improved rates of compliance with these care interventions parallel findings from previous studies.^15,18 Optimization of an institution’s EHR can aid in standardization of care, workflow management, and communication with patients, as well as alert nursing or support staff of screening needs. Techniques such as best practice reminders, patient chart messages, and nursing-entered physician alerts on daily schedules have been shown to increase rates of compliance with nephropathy standards. These findings underscore an additional opportunity for nursing and support staff to be better integrated into care.

Despite the success of this quality improvement initiative, there remain some limitations. The processes we used in this project may not be applicable to every institution and may have limited external validity. Primarily, while these processes may be implemented at some sites, without additional support staff (ie, extra nursing staff, pharmacists) and students to aid in patient outreach, success may be limited due to provider time constraints. Additionally, our workflow process demonstrates significant incorporation of an EHR system for patient outreach. Institutions and/or clinics that heavily rely on paper charts and paper outreach may face barriers with bulk orders (eg, ACR) and messages, interventions that streamlined our population health management. Finally, this project focuses on only 1 aspect of population health management for patients with diabetes. While nephropathy is a critical aspect of caring for individuals with diabetes, this patient outreach does not address retinopathy screening, HbA1c control, or vaccination rates, which are other components of care.

Conclusion

Although this evaluation does not provide insight into why patients were not treated or screened, it demonstrates processes to improve compliance in patients with diabetic nephropathy. Rural health care facilities require an ongoing program of change and evaluation, with the aim to improve the provision of services, increase screening, and encourage team member involvement in health promotion. This study demonstrates that combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy screening and treatment in rural patients with diabetes at a family medicine clinic.

Corresponding author: Amie M. Ashcraft, West Virginia University, Department of Family Medicine, 1 Medical Center Drive, Box 9152, Morgantown, WV 26506; amashcraft@hsc.wvu.edu.

Financial disclosures: None.

Acknowledgment: The authors thank the faculty, residents, nurses, and clinic staff for their hard work and dedication to this effort: Umama Sadia, Michelle Prestoza, Richard Dattola, Greg Doyle, Dana King, Mike Maroon, Kendra Under, Judy Siebert, Christine Snyder, Rachel Burge, Meagan Gribble, Lisa Metts, Kelsey Samek, Sarah Deavers, Amber Kitzmiller, Angela Lamp, Tina Waldeck, and Andrea Sukeruksa.

From West Virginia University, Morgantown, WV.

Abstract

Objective: To describe the strategies a family medicine clinic in Appalachia utilized to increase nephropathy screening rates as well as to explore the factors predictive of nephropathy screening in patients with diabetes.

Design: This quality improvement project targeted the points in the care process when patients are lost to follow-up for nephropathy screening.

Setting and participants: Patients with diabetes cared for by a primary care provider (PCP) at an academic family medicine practice in Appalachia from January 2018 to November 2018.

Interventions: Bulk orders for albumin-to-creatinine (ACR) testing and urine collection during clinic visit, enhanced patient communication through bulk communication reminders and individual patient outreach, and education of clinic providers.

Measurements: Demographic data and monthly nephropathy screening rates.

Results: The nephropathy screening rate increased by 6.2% during the project. Older patients living closer to the clinic who visited their PCP 3 or more times per year were the most likely to be screened.

Conclusion: Combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy treatment and screening in rural patients with diabetes at a family medicine clinic.

Keywords: rural; kidney disease; albumin-to-creatinine ratio; electronic health record.

According to the Centers for Disease Control and Prevention (CDC), an estimated 30.3 million people in the United States—about 9.4% of the population—have been diagnosed with diabetes.¹ Diabetes is the seventh leading cause of death in the United States, and it contributes to other leading causes of death: heart disease and stroke.¹ Diabetes also is related to high morbidity risk and is a leading cause of chronic kidney disease.¹ The total cost of diagnosed diabetes was estimated at $327 billion in direct medical costs and reduced productivity.²

Residents of Appalachia bear a disproportionate burden of diabetes and other related negative health outcomes; these outcomes are influenced by a number of factors, including socioeconomic status, poverty, rurality, and health care access. Rates of chronic disease, such as diabetes, are most pronounced in Appalachia’s most economically distressed counties.^3-5 In 2011, the CDC labeled a 644-county area the “diabetes belt,” which included most of Appalachia.⁶ As a result of this elevated prevalence of diabetes in Appalachia as compared to the rest of the country, complications directly associated with diabetes are more commonly observed in Appalachian residents. One of the most damaging complications is diabetic nephropathy.

Diabetic nephropathy results from damage to the microvasculature of the kidney due to inadequately controlled blood glucose. This, in turn, leads to decreased renal function, eventually leading to clinically significant renal disease. The long-term complications associated with nephropathy can include many comorbid conditions, the most serious of which are progression to end-stage renal disease, dialysis requirement, and early mortality. Diabetic nephropathy affects approximately 40% of patients with type 1 and type 2 diabetes.^7,8

One way to prevent complications of diabetic nephropathy, in addition to good glycemic control in patients with diabetes, is early and regular screening. Currently, the American Diabetes Association (ADA) recommends yearly screening for diabetic nephropathy in the form of a urine albumin-to-creatinine ratio (ACR) for patients 18 to 75 years of age.² This screening to detect diabetic nephropathy is recognized as a marker of quality care by many public and private insurance agencies and medical specialty associations, such as the Centers for Medicare and Medicaid Services.

Many patients with diabetes are cared for by primary care providers (PCP), and these PCP appointments provide an opportune time to screen and appropriately treat nephropathy. Screening opportunities are often missed, however, due to time constraints and competing health priorities. There are also a number of other factors specific to the Appalachian region that reduce the likelihood of screening for diabetic nephropathy, such as a lack of health insurance, the need to travel long distances to see a PCP, work and household responsibilities, low levels of education and health literacy, and a mistrust of outsiders regarding personal matters, including health.^9-11 While nephropathy can have a detrimental impact on patients across populations, it is of particular concern for a state located in the heart of Appalachia, such as West Virginia.

Given the disproportionate burden of diabetes in this region and the potentially severe consequences of undetected nephropathy, clinicians from an academic family medicine clinic in West Virginia undertook a quality improvement project to increase the rate of nephropathy screening and treatment among patients with diabetes. This article describes the intervention strategies the team utilized to increase nephropathy screening and treatment in patients 18 to 75 years of age who met quality measures for nephropathy screening or treatment in the previous 12 months and explores the factors most predictive of nephropathy screening in Appalachian patients in this age group. It also reports the challenges and opportunities encountered and offers suggestions for other providers and clinics attempting to increase their nephropathy screening rates.

Methods

Setting and Study Population

The study population included patients ages 18 to 75 years under the care of providers in an academic family medicine practice in West Virginia who had been diagnosed with diabetes mellitus. The study focused on those patients overdue for diabetic nephropathy screening (ie, had not been screened in previous 12 months). The project began in January 2018 with a screening rate of 83.8%. The goal of this project was to increase this compliance metric by at least 5%. The project protocol was submitted to the West Virginia University Institutional Review Board, and, because it is a quality improvement project, permission was given to proceed without a board review.

Interventions

The team identified and implemented several interventions intended to reduce screening barriers and increase the screening rate.

Bulk orders for ACR and urine collection during clinic visits. Prior to initiation of this project, it was left to individual clinic providers to order nephropathy screening for patients with diabetes during a clinic visit; after receiving the order for “random urine microalbumin/creatinine ratio,” patients then had to travel to a lab to provide a urine sample. For this project and moving forward, the team changed to the procedure of initiating bulk ACR orders and collecting urine samples during clinic visits from all patients ages 18 to 75 years who have diabetes.

Bulk communication reminders. Since many patients with diabetes may not have realized they were overdue for nephropathy screening, the team began sending out bulk communication reminders through either the institution’s electronic health record (EHR; MyChart) or postal service–delivered physical letters (according to patient communication preferences) to remind patients that they were due for screening and to encourage them to schedule an appointment or keep a previously scheduled appointment with their PCP.

Individual patient outreach. A team of pharmacy students led by a licensed pharmacist in the family medicine clinic contacted patients overdue for screening even after bulk communication reminders went out. The students telephoned patients 2 to 3 months following the bulk communication. The students obtained an updated list of patients with diabetes ages 18 to 75 years from an EHR quality report. They began by prescreening the patients on the overdue list for potential candidacy for an angiotensin-converting enzyme (ACE) inhibitor or an angiotensin II receptor blocker (ARB). Screening for candidacy included evaluation of recent blood pressure readings, electrolytes (ie, basic metabolic panel), and ACR. If the students determined a patient was a candidate, they presented the patient to the preceptor for verification and then reached out to the provider with a recommendation. If the provider agreed, the student contacted the patient by telephone for medication counseling and education. The remaining patients determined not to be candidates for ACE inhibitors or ARBs were contacted by the pharmacy students by telephone to remind them that laboratory work was pending. Up to 3 phone call attempts were made before patients were determined to be unreachable. Students left voice mails with generic reminders if a patient could not be reached. If a patient answered, the student provided a reminder but also reviewed indications for lab work, the reason why the provider wished for follow-up, and updated lab hours. Students also followed up with the results of the work-up, as appropriate. During this outreach process, the student team encountered a number of patients who had moved or changed to a PCP outside of the family medicine clinic. In these cases, the EHR was updated and those patients were removed from the list of patients altogether.

Education of clinic providers. Clinic providers were educated during faculty and resident meetings and didactic learning sessions on identifying patients within the EHR who are due for nephropathy screening. They also received instruction on how to update the EHR to reflect completed screenings.

Data Analysis

All analyses in this study were conducted using SAS (version 9.4, 2013, SAS Institute Inc., Cary, NC). Descriptive analyses were conducted to summarize basic patient demographic information. To compare patients screened within the previous 12 months to those patients overdue for screening, 2-sample t-tests were used to examine differences in patients’ age, HbA1c, ACR, and creatinine level and the distance (in miles) between the patient’s home and the clinic. Chi-square analyses were used to examine the relationship between whether a patient was recently screened for nephropathy and the patient’s insurance, number of patient visits in the previous 12 months, and provider level. Logistic regression analyses were conducted to control for covariates and to explore which factors were most predictive of nephropathy screening. All tests were 2-tailed, and P values less than 0.05 were considered statistically significant.

Patient Characteristics

There were 1676 family medicine clinic patients with diabetes between 18 and 75 years of age (Table 1 and Table 2). Of the total sample, 1489 (88.8%) had completed screening for nephropathy in the 12 months prior to evaluation, and 67.5%, 23.7%, and 8.8% of patients had private insurance, Medicare, and Medicaid, respectively.

The mean (SD) age of the patients was 56.3 (11.9) years. The mean distance between the patient’s home and the clinic was 26.6 (76.8) miles. The mean number of visits was 3.6 (2.9) per year, and 43.0% of the patientvisited the clinic more than 3 times in a year. The mean values for HbA1c (%), creatinine (g/mol), and ACR (mg/g) were 7.7 (1.9), 1.0 (0.7), and 9.4 (31.4), respectively.

Screening of Patients for Nephropathy

Patients with Medicare and private insurance were more likely to have completed the nephropathy screening than those with Medicaid (92.5% versus 88.8% versus 82.8%, P = 0.004; Table 3 and Table 4).

Changes in Screening Rate

The practice-wide screening rate was 83.8% at the start of this project in January 2018. The screening rate steadily increased throughout 2018, reaching 90.3% in August 2018, and then leveled off around 90% when the project was concluded at the end of November 2018 (Figure). As an added benefit of the increased screening rates, a number of patients were initiated on an ACE inhibitor or ARB based on the team’s screening efforts.

Predictors of Nephropathy Screening

A logistic regression analysis was conducted with nephropathy screening (screened or not screened) as the outcome and 7 patient characteristics as predictors: type of insurance (private, Medicare, or Medicaid), PCP visits in the past 12 months (≤ 3 or > 3), distance in miles of the patient’s residence from the clinic, age, last HbA1c value, last ACR value, and last creatinine value. A test of the full model with all 7 predictors was statistically significant (χ² (8) = 57.77, P < 0.001). Table 5 shows regression coefficients, Wald statistics, and 95% confidence intervals for odds ratios for each of the 7 predictors. According to the Wald criterion, 3 patient characteristics were significant predictors of nephropathy screening: age, distance between the patient’s home and clinic, and number of PCP visits in the past 12 months. After adjusting for the covariates, there were still significant associations between the nephropathy screening status and age ( χ²(1) = 9.64, P < 0.01); distance between the patient’s home and the clinic (χ²(1) = 3.98, P < 0.05); and the number of PCP visits in the previous year (χ²(1) = 21.74, P < 0.001). With each 1-year increment in age, the odds of completing the nephropathy screening increased by 3.2%. With each 1-mile increase in the distance between the patient’s home and clinic, the odds of completing the nephropathy screening decreased by 0.2%. Patients who visited the clinic more than 3 times in a year were 3.9 times (95% confidence interval, 2.2-7.0) more likely to complete the nephropathy screening than their counterparts who visited fewer than 3 times per year.

In summary, older patients living within about 164 miles of the clinic (ie, within 1 standard deviation from the average miles between patient’s homes and the clinic) who visited their PCP 3 or more times per year were the most likely to be screened.

Diabetic nephropathy is a critical issue facing family medicine providers and patients. The morbidity and mortality costs are significant, as diabetic nephropathy is the leading cause of end-stage renal disease. While the ADA recommends annual ACR screening in patients with diabetes and prescription of ACE inhibitors or ARBs in patients who qualify, many patients do not receive these interventions, despite following up with a provider.^12-15 There is no current literature that indicates the compliance rates in the rural setting. Due to health disparities in the rural setting noted in the literature, it could be hypothesized that these individuals are at high risk of not meeting these screening and treatment recommendations.^16,17 Limited access to care and resources, gaps in insurance coverage, and lower health literacy are a few barriers identified in the rural population that may influence whether these measures are met.¹⁷ 

Considering the disease burden of diabetes and its related complications, including nephropathy, consistent screening is necessary to reduce diabetes-related burdens and cost, while also increasing the quality of life for patients with diabetes. All parties must be involved to ensure appropriate compliance and treatment. Our institution’s implementation of quality improvement strategies has key implications for nephropathy screening and treatment efforts in rural settings.

An additional step of having a health care provider (other than the PCP) screen all patients who are not meeting the standard allows for identification of gaps in care. In our quality improvement workflow, the clinical pharmacist screened all patients for candidacy for ACE inhibitor/ARB therapy. While only a small percentage of patients qualified, many of these patients had previously been on therapy and were discontinued for an unknown reason or were stopped due to an acute condition (eg, acute kidney injury) and never restarted after recovery. Other patients required additional education that therapy would be utilized for nephroprotection versus blood pressure management (secondary to an elevated ACR). This highlights the importance of transitions of care and ongoing, intensive education, not only during initial diagnosis but also throughout the disease-state progression.

Utilization of EHRs and telephone outreach are additional aspects of care that can be provided. Our improved rates of compliance with these care interventions parallel findings from previous studies.^15,18 Optimization of an institution’s EHR can aid in standardization of care, workflow management, and communication with patients, as well as alert nursing or support staff of screening needs. Techniques such as best practice reminders, patient chart messages, and nursing-entered physician alerts on daily schedules have been shown to increase rates of compliance with nephropathy standards. These findings underscore an additional opportunity for nursing and support staff to be better integrated into care.

Despite the success of this quality improvement initiative, there remain some limitations. The processes we used in this project may not be applicable to every institution and may have limited external validity. Primarily, while these processes may be implemented at some sites, without additional support staff (ie, extra nursing staff, pharmacists) and students to aid in patient outreach, success may be limited due to provider time constraints. Additionally, our workflow process demonstrates significant incorporation of an EHR system for patient outreach. Institutions and/or clinics that heavily rely on paper charts and paper outreach may face barriers with bulk orders (eg, ACR) and messages, interventions that streamlined our population health management. Finally, this project focuses on only 1 aspect of population health management for patients with diabetes. While nephropathy is a critical aspect of caring for individuals with diabetes, this patient outreach does not address retinopathy screening, HbA1c control, or vaccination rates, which are other components of care.

Conclusion

Although this evaluation does not provide insight into why patients were not treated or screened, it demonstrates processes to improve compliance in patients with diabetic nephropathy. Rural health care facilities require an ongoing program of change and evaluation, with the aim to improve the provision of services, increase screening, and encourage team member involvement in health promotion. This study demonstrates that combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy screening and treatment in rural patients with diabetes at a family medicine clinic.

Corresponding author: Amie M. Ashcraft, West Virginia University, Department of Family Medicine, 1 Medical Center Drive, Box 9152, Morgantown, WV 26506; amashcraft@hsc.wvu.edu.

Financial disclosures: None.

Acknowledgment: The authors thank the faculty, residents, nurses, and clinic staff for their hard work and dedication to this effort: Umama Sadia, Michelle Prestoza, Richard Dattola, Greg Doyle, Dana King, Mike Maroon, Kendra Under, Judy Siebert, Christine Snyder, Rachel Burge, Meagan Gribble, Lisa Metts, Kelsey Samek, Sarah Deavers, Amber Kitzmiller, Angela Lamp, Tina Waldeck, and Andrea Sukeruksa.

From West Virginia University, Morgantown, WV.

Abstract

Objective: To describe the strategies a family medicine clinic in Appalachia utilized to increase nephropathy screening rates as well as to explore the factors predictive of nephropathy screening in patients with diabetes.

Design: This quality improvement project targeted the points in the care process when patients are lost to follow-up for nephropathy screening.

Setting and participants: Patients with diabetes cared for by a primary care provider (PCP) at an academic family medicine practice in Appalachia from January 2018 to November 2018.

Interventions: Bulk orders for albumin-to-creatinine (ACR) testing and urine collection during clinic visit, enhanced patient communication through bulk communication reminders and individual patient outreach, and education of clinic providers.

Measurements: Demographic data and monthly nephropathy screening rates.

Results: The nephropathy screening rate increased by 6.2% during the project. Older patients living closer to the clinic who visited their PCP 3 or more times per year were the most likely to be screened.

Conclusion: Combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy treatment and screening in rural patients with diabetes at a family medicine clinic.

Keywords: rural; kidney disease; albumin-to-creatinine ratio; electronic health record.

According to the Centers for Disease Control and Prevention (CDC), an estimated 30.3 million people in the United States—about 9.4% of the population—have been diagnosed with diabetes.¹ Diabetes is the seventh leading cause of death in the United States, and it contributes to other leading causes of death: heart disease and stroke.¹ Diabetes also is related to high morbidity risk and is a leading cause of chronic kidney disease.¹ The total cost of diagnosed diabetes was estimated at $327 billion in direct medical costs and reduced productivity.²

Residents of Appalachia bear a disproportionate burden of diabetes and other related negative health outcomes; these outcomes are influenced by a number of factors, including socioeconomic status, poverty, rurality, and health care access. Rates of chronic disease, such as diabetes, are most pronounced in Appalachia’s most economically distressed counties.^3-5 In 2011, the CDC labeled a 644-county area the “diabetes belt,” which included most of Appalachia.⁶ As a result of this elevated prevalence of diabetes in Appalachia as compared to the rest of the country, complications directly associated with diabetes are more commonly observed in Appalachian residents. One of the most damaging complications is diabetic nephropathy.

Diabetic nephropathy results from damage to the microvasculature of the kidney due to inadequately controlled blood glucose. This, in turn, leads to decreased renal function, eventually leading to clinically significant renal disease. The long-term complications associated with nephropathy can include many comorbid conditions, the most serious of which are progression to end-stage renal disease, dialysis requirement, and early mortality. Diabetic nephropathy affects approximately 40% of patients with type 1 and type 2 diabetes.^7,8

One way to prevent complications of diabetic nephropathy, in addition to good glycemic control in patients with diabetes, is early and regular screening. Currently, the American Diabetes Association (ADA) recommends yearly screening for diabetic nephropathy in the form of a urine albumin-to-creatinine ratio (ACR) for patients 18 to 75 years of age.² This screening to detect diabetic nephropathy is recognized as a marker of quality care by many public and private insurance agencies and medical specialty associations, such as the Centers for Medicare and Medicaid Services.

Many patients with diabetes are cared for by primary care providers (PCP), and these PCP appointments provide an opportune time to screen and appropriately treat nephropathy. Screening opportunities are often missed, however, due to time constraints and competing health priorities. There are also a number of other factors specific to the Appalachian region that reduce the likelihood of screening for diabetic nephropathy, such as a lack of health insurance, the need to travel long distances to see a PCP, work and household responsibilities, low levels of education and health literacy, and a mistrust of outsiders regarding personal matters, including health.^9-11 While nephropathy can have a detrimental impact on patients across populations, it is of particular concern for a state located in the heart of Appalachia, such as West Virginia.

Given the disproportionate burden of diabetes in this region and the potentially severe consequences of undetected nephropathy, clinicians from an academic family medicine clinic in West Virginia undertook a quality improvement project to increase the rate of nephropathy screening and treatment among patients with diabetes. This article describes the intervention strategies the team utilized to increase nephropathy screening and treatment in patients 18 to 75 years of age who met quality measures for nephropathy screening or treatment in the previous 12 months and explores the factors most predictive of nephropathy screening in Appalachian patients in this age group. It also reports the challenges and opportunities encountered and offers suggestions for other providers and clinics attempting to increase their nephropathy screening rates.

Methods

Setting and Study Population

The study population included patients ages 18 to 75 years under the care of providers in an academic family medicine practice in West Virginia who had been diagnosed with diabetes mellitus. The study focused on those patients overdue for diabetic nephropathy screening (ie, had not been screened in previous 12 months). The project began in January 2018 with a screening rate of 83.8%. The goal of this project was to increase this compliance metric by at least 5%. The project protocol was submitted to the West Virginia University Institutional Review Board, and, because it is a quality improvement project, permission was given to proceed without a board review.

Interventions

The team identified and implemented several interventions intended to reduce screening barriers and increase the screening rate.

Bulk orders for ACR and urine collection during clinic visits. Prior to initiation of this project, it was left to individual clinic providers to order nephropathy screening for patients with diabetes during a clinic visit; after receiving the order for “random urine microalbumin/creatinine ratio,” patients then had to travel to a lab to provide a urine sample. For this project and moving forward, the team changed to the procedure of initiating bulk ACR orders and collecting urine samples during clinic visits from all patients ages 18 to 75 years who have diabetes.

Bulk communication reminders. Since many patients with diabetes may not have realized they were overdue for nephropathy screening, the team began sending out bulk communication reminders through either the institution’s electronic health record (EHR; MyChart) or postal service–delivered physical letters (according to patient communication preferences) to remind patients that they were due for screening and to encourage them to schedule an appointment or keep a previously scheduled appointment with their PCP.

Individual patient outreach. A team of pharmacy students led by a licensed pharmacist in the family medicine clinic contacted patients overdue for screening even after bulk communication reminders went out. The students telephoned patients 2 to 3 months following the bulk communication. The students obtained an updated list of patients with diabetes ages 18 to 75 years from an EHR quality report. They began by prescreening the patients on the overdue list for potential candidacy for an angiotensin-converting enzyme (ACE) inhibitor or an angiotensin II receptor blocker (ARB). Screening for candidacy included evaluation of recent blood pressure readings, electrolytes (ie, basic metabolic panel), and ACR. If the students determined a patient was a candidate, they presented the patient to the preceptor for verification and then reached out to the provider with a recommendation. If the provider agreed, the student contacted the patient by telephone for medication counseling and education. The remaining patients determined not to be candidates for ACE inhibitors or ARBs were contacted by the pharmacy students by telephone to remind them that laboratory work was pending. Up to 3 phone call attempts were made before patients were determined to be unreachable. Students left voice mails with generic reminders if a patient could not be reached. If a patient answered, the student provided a reminder but also reviewed indications for lab work, the reason why the provider wished for follow-up, and updated lab hours. Students also followed up with the results of the work-up, as appropriate. During this outreach process, the student team encountered a number of patients who had moved or changed to a PCP outside of the family medicine clinic. In these cases, the EHR was updated and those patients were removed from the list of patients altogether.

Education of clinic providers. Clinic providers were educated during faculty and resident meetings and didactic learning sessions on identifying patients within the EHR who are due for nephropathy screening. They also received instruction on how to update the EHR to reflect completed screenings.

Data Analysis

All analyses in this study were conducted using SAS (version 9.4, 2013, SAS Institute Inc., Cary, NC). Descriptive analyses were conducted to summarize basic patient demographic information. To compare patients screened within the previous 12 months to those patients overdue for screening, 2-sample t-tests were used to examine differences in patients’ age, HbA1c, ACR, and creatinine level and the distance (in miles) between the patient’s home and the clinic. Chi-square analyses were used to examine the relationship between whether a patient was recently screened for nephropathy and the patient’s insurance, number of patient visits in the previous 12 months, and provider level. Logistic regression analyses were conducted to control for covariates and to explore which factors were most predictive of nephropathy screening. All tests were 2-tailed, and P values less than 0.05 were considered statistically significant.

Patient Characteristics

There were 1676 family medicine clinic patients with diabetes between 18 and 75 years of age (Table 1 and Table 2). Of the total sample, 1489 (88.8%) had completed screening for nephropathy in the 12 months prior to evaluation, and 67.5%, 23.7%, and 8.8% of patients had private insurance, Medicare, and Medicaid, respectively.

The mean (SD) age of the patients was 56.3 (11.9) years. The mean distance between the patient’s home and the clinic was 26.6 (76.8) miles. The mean number of visits was 3.6 (2.9) per year, and 43.0% of the patientvisited the clinic more than 3 times in a year. The mean values for HbA1c (%), creatinine (g/mol), and ACR (mg/g) were 7.7 (1.9), 1.0 (0.7), and 9.4 (31.4), respectively.

Screening of Patients for Nephropathy

Patients with Medicare and private insurance were more likely to have completed the nephropathy screening than those with Medicaid (92.5% versus 88.8% versus 82.8%, P = 0.004; Table 3 and Table 4).

Changes in Screening Rate

The practice-wide screening rate was 83.8% at the start of this project in January 2018. The screening rate steadily increased throughout 2018, reaching 90.3% in August 2018, and then leveled off around 90% when the project was concluded at the end of November 2018 (Figure). As an added benefit of the increased screening rates, a number of patients were initiated on an ACE inhibitor or ARB based on the team’s screening efforts.

Predictors of Nephropathy Screening

A logistic regression analysis was conducted with nephropathy screening (screened or not screened) as the outcome and 7 patient characteristics as predictors: type of insurance (private, Medicare, or Medicaid), PCP visits in the past 12 months (≤ 3 or > 3), distance in miles of the patient’s residence from the clinic, age, last HbA1c value, last ACR value, and last creatinine value. A test of the full model with all 7 predictors was statistically significant (χ² (8) = 57.77, P < 0.001). Table 5 shows regression coefficients, Wald statistics, and 95% confidence intervals for odds ratios for each of the 7 predictors. According to the Wald criterion, 3 patient characteristics were significant predictors of nephropathy screening: age, distance between the patient’s home and clinic, and number of PCP visits in the past 12 months. After adjusting for the covariates, there were still significant associations between the nephropathy screening status and age ( χ²(1) = 9.64, P < 0.01); distance between the patient’s home and the clinic (χ²(1) = 3.98, P < 0.05); and the number of PCP visits in the previous year (χ²(1) = 21.74, P < 0.001). With each 1-year increment in age, the odds of completing the nephropathy screening increased by 3.2%. With each 1-mile increase in the distance between the patient’s home and clinic, the odds of completing the nephropathy screening decreased by 0.2%. Patients who visited the clinic more than 3 times in a year were 3.9 times (95% confidence interval, 2.2-7.0) more likely to complete the nephropathy screening than their counterparts who visited fewer than 3 times per year.

In summary, older patients living within about 164 miles of the clinic (ie, within 1 standard deviation from the average miles between patient’s homes and the clinic) who visited their PCP 3 or more times per year were the most likely to be screened.

Diabetic nephropathy is a critical issue facing family medicine providers and patients. The morbidity and mortality costs are significant, as diabetic nephropathy is the leading cause of end-stage renal disease. While the ADA recommends annual ACR screening in patients with diabetes and prescription of ACE inhibitors or ARBs in patients who qualify, many patients do not receive these interventions, despite following up with a provider.^12-15 There is no current literature that indicates the compliance rates in the rural setting. Due to health disparities in the rural setting noted in the literature, it could be hypothesized that these individuals are at high risk of not meeting these screening and treatment recommendations.^16,17 Limited access to care and resources, gaps in insurance coverage, and lower health literacy are a few barriers identified in the rural population that may influence whether these measures are met.¹⁷ 

Considering the disease burden of diabetes and its related complications, including nephropathy, consistent screening is necessary to reduce diabetes-related burdens and cost, while also increasing the quality of life for patients with diabetes. All parties must be involved to ensure appropriate compliance and treatment. Our institution’s implementation of quality improvement strategies has key implications for nephropathy screening and treatment efforts in rural settings.

An additional step of having a health care provider (other than the PCP) screen all patients who are not meeting the standard allows for identification of gaps in care. In our quality improvement workflow, the clinical pharmacist screened all patients for candidacy for ACE inhibitor/ARB therapy. While only a small percentage of patients qualified, many of these patients had previously been on therapy and were discontinued for an unknown reason or were stopped due to an acute condition (eg, acute kidney injury) and never restarted after recovery. Other patients required additional education that therapy would be utilized for nephroprotection versus blood pressure management (secondary to an elevated ACR). This highlights the importance of transitions of care and ongoing, intensive education, not only during initial diagnosis but also throughout the disease-state progression.

Utilization of EHRs and telephone outreach are additional aspects of care that can be provided. Our improved rates of compliance with these care interventions parallel findings from previous studies.^15,18 Optimization of an institution’s EHR can aid in standardization of care, workflow management, and communication with patients, as well as alert nursing or support staff of screening needs. Techniques such as best practice reminders, patient chart messages, and nursing-entered physician alerts on daily schedules have been shown to increase rates of compliance with nephropathy standards. These findings underscore an additional opportunity for nursing and support staff to be better integrated into care.

Despite the success of this quality improvement initiative, there remain some limitations. The processes we used in this project may not be applicable to every institution and may have limited external validity. Primarily, while these processes may be implemented at some sites, without additional support staff (ie, extra nursing staff, pharmacists) and students to aid in patient outreach, success may be limited due to provider time constraints. Additionally, our workflow process demonstrates significant incorporation of an EHR system for patient outreach. Institutions and/or clinics that heavily rely on paper charts and paper outreach may face barriers with bulk orders (eg, ACR) and messages, interventions that streamlined our population health management. Finally, this project focuses on only 1 aspect of population health management for patients with diabetes. While nephropathy is a critical aspect of caring for individuals with diabetes, this patient outreach does not address retinopathy screening, HbA1c control, or vaccination rates, which are other components of care.

Conclusion

Although this evaluation does not provide insight into why patients were not treated or screened, it demonstrates processes to improve compliance in patients with diabetic nephropathy. Rural health care facilities require an ongoing program of change and evaluation, with the aim to improve the provision of services, increase screening, and encourage team member involvement in health promotion. This study demonstrates that combining team-based interventions with quality control monitoring can significantly improve compliance with recommended nephropathy screening and treatment in rural patients with diabetes at a family medicine clinic.

Corresponding author: Amie M. Ashcraft, West Virginia University, Department of Family Medicine, 1 Medical Center Drive, Box 9152, Morgantown, WV 26506; amashcraft@hsc.wvu.edu.

Financial disclosures: None.

Acknowledgment: The authors thank the faculty, residents, nurses, and clinic staff for their hard work and dedication to this effort: Umama Sadia, Michelle Prestoza, Richard Dattola, Greg Doyle, Dana King, Mike Maroon, Kendra Under, Judy Siebert, Christine Snyder, Rachel Burge, Meagan Gribble, Lisa Metts, Kelsey Samek, Sarah Deavers, Amber Kitzmiller, Angela Lamp, Tina Waldeck, and Andrea Sukeruksa.

Study Overview

Objective. To examine the effectiveness of a hub site–based care delivery system in delivering a dementia care management program to persons with dementia and their caregivers.

Design. Randomized pragmatic clinical trial enrolling dyads of persons with dementia and their caregiver. Study participants were randomly assigned to the dementia care management program and usual care in a 2:1 ratio.

Setting and participants. The study was conducted from 2 hub sites: the University of California, San Francisco, and the University of Nebraska Medical Center in Omaha. Each hub-site team served persons with dementia and their caregivers in California, Nebraska, and Iowa in both urban and rural areas. Participants were recruited through referral by treating providers or self-referral in response to advertising presented through a community outreach event, in the news, or on the internet. Eligibility requirements included: having a dementia diagnosis made by a treating provider; age older than 45 years; Medicare or Medicaid enrollment or eligibility; presence of a caregiver willing to enroll in the study; fluency in English, Spanish, or Cantonese; and residence in California, Nebraska, or Iowa. Exclusion criteria included residence in a nursing home. Out of 2585 referred dyads of persons with dementia and caregivers, 780 met inclusion criteria and were enrolled. A 2:1 randomization yielded 512 dyads in the intervention group and 268 dyads in the control group.

Intervention. The dementia care management program was implemented through the Care Ecosystem, a telephone- and internet-based supportive care intervention delivered by care team navigators. The navigators were unlicensed but trained dementia care guides working under the supervision of an advanced practice nurse, social worker, and pharmacist. The intervention consisted of telephone calls, monthly or at a frequency determined by needs and preferences, placed by navigators over a 12-month period; the content of the calls included response to immediate needs of persons with dementia and their caregiver, screening for common problems, and provision of support and education using care plan protocols. Caregivers and persons with dementia were encouraged to initiate contact through email, mail, or telephone for dementia-related questions. Additional support was provided by an advanced practice nurse, social worker, or pharmacist, as needed, and these health care professionals conducted further communication with the persons with dementia, caregiver, or outside professionals, such as physicians, for the persons with dementia, as needed. The average number of telephone calls over the 12-month period was 15.3 (standard deviation, 11.3). Participants assigned to usual care were offered contact information on dementia and aging-related organizations, including the Alzheimer’s Association and the Area Agencies on Aging, and also were sent a quarterly newsletter with general information about dementia.

Main outcome measures. The primary outcome measure was the Quality of Life in Alzheimer’s Disease score obtained by caregiver interview. This quality of life measure includes the following aspects, each rated on an ordinal scale of 1 to 4: physical health, energy level, mood, living situation, memory, family, closest relationship, friends, self, ability to do things for fun, finances, and life as a whole. The scores range from 13 to 52, with a higher score indicating better quality of life for persons with dementia. Other outcomes included frequency of emergency room visits, hospital use, and ambulance use; caregiver depression score from the Patient Health Questionnaire scale; caregiver burden score using the 12-item Zarit Burden Interview; caregiver self-efficacy; and caregiver satisfaction.

Main results. The study found that the quality of life for persons with dementia declined more in the usual care group than in the intervention group during the 12-month study period (difference of 0.53; 95% confidence interval, 0.25-1.3; P = 0.04). Persons with dementia also had fewer emergency room visits, with a number needed to treat to prevent 1 emergency room visit of 5. The intervention did not reduce ambulance use or hospital use. Caregivers in the intervention group had a greater decline in depression when compared to usual care; the frequency of moderate to severe depression decreased from 13.4% at baseline to 7.9% at 12 months (P = 0.004). Caregiver burden declined more in the intervention group than in the control group at 12 months (P = 0.046). In terms of caregiver satisfaction, 97% of caregivers surveyed in the intervention group said they would recommend the intervention to another caregiver; 45% indicated they were very satisfied, and 33% that they were satisfied.

Conclusion. Delivering dementia care via telephone and internet through a collaborative program with care navigators can improve caregiver burden and well-being and improve quality of life, emergency room utilization, and depression for persons with dementia. In addition, the program was well received.

Commentary

Dementia, including Alzheimer’s disease, primarily affects older adults and is characterized by declines in memory and cognitive function. It is often accompanied by neuropsychological symptoms such as agitation, wandering, and physical and verbal outbursts, which are debilitating for persons living with dementia and difficult to cope with for caregivers.¹ These symptoms are often the source of caregiver stress, potentially leading to caregiver depression and eventual need for long-term institution-based care, such as nursing home placement.²

Prior literature has established the potential effect of support in improving caregiver outcomes, including caregiver stress and burden, through interventions such as enhancing resources for caregivers, teaching coping strategies to caregivers, and teaching caregivers how to manage support for their loved ones.^3,4 However, wider adoption of these interventions may be limited if the interventions involve in-person meetings or activities that take caregivers away from caregiving; the scalability of these programs is also limited by their ability to reach persons with dementia and their caregivers. These barriers are particularly important for older adults living in rural areas, where the availability of resources and distance from access to quality care may be particularly limiting.⁵ Leveraging advances in technology and telecommunication, this study examined the effects of providing dementia care support via telephone and internet using a trained, unlicensed care navigator as the main point of contact. The results showed improved quality of life for persons with dementia, reduced need for emergency room visits, and reduced caregiver burden and depression. The intervention is promising as a scalable intervention that may impact dementia care nationwide.

Despite the promising results, there are several issues regarding the intervention’s applicability and impact that future studies may help to further clarify. Although the improvement in quality of life in persons with dementia is important to document, it is unclear whether this difference is clinically significant. Also, it may be important to examine whether the 12-month program has sustained impact beyond the study period, although the intervention could be conceived as a long-term care solution. If the intervention is sustained beyond 12 months, future studies may look at other clinical outcomes, such as incidence of institutionalization and perhaps time to institutionalization. The study population consisted of persons with dementia of various stages, half of whom had mild disease. Future studies may further clarify at which stage of dementia the intervention is most useful. Other changes that occurred during the study period, such as change in the use of paid home-based support services and referrals to other relevant evaluations and treatment, may provide further clues about how the dementia care intervention achieved its beneficial effects.

Applications for Clinical Practice

From the health systems perspective, dementia care accounts for significant resources, and these costs are expected to grow as the population ages and dementia prevalence increases. Identifying potentially scalable interventions that yield clinical benefits and are sustainable from a cost perspective is an important step forward in improving care for persons with dementia and their caregivers across the nation. The use of centralized hubs to deliver this intervention and the novel use of telecommunications advances make this intervention applicable across large areas. Policy makers should explore how an intervention such as this could be established and sustained in our health care system.

–William W. Hung, MD, MPH

Study Overview

Objective. To examine the effectiveness of a hub site–based care delivery system in delivering a dementia care management program to persons with dementia and their caregivers.

Design. Randomized pragmatic clinical trial enrolling dyads of persons with dementia and their caregiver. Study participants were randomly assigned to the dementia care management program and usual care in a 2:1 ratio.

Setting and participants. The study was conducted from 2 hub sites: the University of California, San Francisco, and the University of Nebraska Medical Center in Omaha. Each hub-site team served persons with dementia and their caregivers in California, Nebraska, and Iowa in both urban and rural areas. Participants were recruited through referral by treating providers or self-referral in response to advertising presented through a community outreach event, in the news, or on the internet. Eligibility requirements included: having a dementia diagnosis made by a treating provider; age older than 45 years; Medicare or Medicaid enrollment or eligibility; presence of a caregiver willing to enroll in the study; fluency in English, Spanish, or Cantonese; and residence in California, Nebraska, or Iowa. Exclusion criteria included residence in a nursing home. Out of 2585 referred dyads of persons with dementia and caregivers, 780 met inclusion criteria and were enrolled. A 2:1 randomization yielded 512 dyads in the intervention group and 268 dyads in the control group.

Intervention. The dementia care management program was implemented through the Care Ecosystem, a telephone- and internet-based supportive care intervention delivered by care team navigators. The navigators were unlicensed but trained dementia care guides working under the supervision of an advanced practice nurse, social worker, and pharmacist. The intervention consisted of telephone calls, monthly or at a frequency determined by needs and preferences, placed by navigators over a 12-month period; the content of the calls included response to immediate needs of persons with dementia and their caregiver, screening for common problems, and provision of support and education using care plan protocols. Caregivers and persons with dementia were encouraged to initiate contact through email, mail, or telephone for dementia-related questions. Additional support was provided by an advanced practice nurse, social worker, or pharmacist, as needed, and these health care professionals conducted further communication with the persons with dementia, caregiver, or outside professionals, such as physicians, for the persons with dementia, as needed. The average number of telephone calls over the 12-month period was 15.3 (standard deviation, 11.3). Participants assigned to usual care were offered contact information on dementia and aging-related organizations, including the Alzheimer’s Association and the Area Agencies on Aging, and also were sent a quarterly newsletter with general information about dementia.

Main outcome measures. The primary outcome measure was the Quality of Life in Alzheimer’s Disease score obtained by caregiver interview. This quality of life measure includes the following aspects, each rated on an ordinal scale of 1 to 4: physical health, energy level, mood, living situation, memory, family, closest relationship, friends, self, ability to do things for fun, finances, and life as a whole. The scores range from 13 to 52, with a higher score indicating better quality of life for persons with dementia. Other outcomes included frequency of emergency room visits, hospital use, and ambulance use; caregiver depression score from the Patient Health Questionnaire scale; caregiver burden score using the 12-item Zarit Burden Interview; caregiver self-efficacy; and caregiver satisfaction.

Main results. The study found that the quality of life for persons with dementia declined more in the usual care group than in the intervention group during the 12-month study period (difference of 0.53; 95% confidence interval, 0.25-1.3; P = 0.04). Persons with dementia also had fewer emergency room visits, with a number needed to treat to prevent 1 emergency room visit of 5. The intervention did not reduce ambulance use or hospital use. Caregivers in the intervention group had a greater decline in depression when compared to usual care; the frequency of moderate to severe depression decreased from 13.4% at baseline to 7.9% at 12 months (P = 0.004). Caregiver burden declined more in the intervention group than in the control group at 12 months (P = 0.046). In terms of caregiver satisfaction, 97% of caregivers surveyed in the intervention group said they would recommend the intervention to another caregiver; 45% indicated they were very satisfied, and 33% that they were satisfied.

Conclusion. Delivering dementia care via telephone and internet through a collaborative program with care navigators can improve caregiver burden and well-being and improve quality of life, emergency room utilization, and depression for persons with dementia. In addition, the program was well received.

Commentary

Dementia, including Alzheimer’s disease, primarily affects older adults and is characterized by declines in memory and cognitive function. It is often accompanied by neuropsychological symptoms such as agitation, wandering, and physical and verbal outbursts, which are debilitating for persons living with dementia and difficult to cope with for caregivers.¹ These symptoms are often the source of caregiver stress, potentially leading to caregiver depression and eventual need for long-term institution-based care, such as nursing home placement.²

Prior literature has established the potential effect of support in improving caregiver outcomes, including caregiver stress and burden, through interventions such as enhancing resources for caregivers, teaching coping strategies to caregivers, and teaching caregivers how to manage support for their loved ones.^3,4 However, wider adoption of these interventions may be limited if the interventions involve in-person meetings or activities that take caregivers away from caregiving; the scalability of these programs is also limited by their ability to reach persons with dementia and their caregivers. These barriers are particularly important for older adults living in rural areas, where the availability of resources and distance from access to quality care may be particularly limiting.⁵ Leveraging advances in technology and telecommunication, this study examined the effects of providing dementia care support via telephone and internet using a trained, unlicensed care navigator as the main point of contact. The results showed improved quality of life for persons with dementia, reduced need for emergency room visits, and reduced caregiver burden and depression. The intervention is promising as a scalable intervention that may impact dementia care nationwide.

Despite the promising results, there are several issues regarding the intervention’s applicability and impact that future studies may help to further clarify. Although the improvement in quality of life in persons with dementia is important to document, it is unclear whether this difference is clinically significant. Also, it may be important to examine whether the 12-month program has sustained impact beyond the study period, although the intervention could be conceived as a long-term care solution. If the intervention is sustained beyond 12 months, future studies may look at other clinical outcomes, such as incidence of institutionalization and perhaps time to institutionalization. The study population consisted of persons with dementia of various stages, half of whom had mild disease. Future studies may further clarify at which stage of dementia the intervention is most useful. Other changes that occurred during the study period, such as change in the use of paid home-based support services and referrals to other relevant evaluations and treatment, may provide further clues about how the dementia care intervention achieved its beneficial effects.

Applications for Clinical Practice

From the health systems perspective, dementia care accounts for significant resources, and these costs are expected to grow as the population ages and dementia prevalence increases. Identifying potentially scalable interventions that yield clinical benefits and are sustainable from a cost perspective is an important step forward in improving care for persons with dementia and their caregivers across the nation. The use of centralized hubs to deliver this intervention and the novel use of telecommunications advances make this intervention applicable across large areas. Policy makers should explore how an intervention such as this could be established and sustained in our health care system.

–William W. Hung, MD, MPH

Study Overview

Objective. To examine the effectiveness of a hub site–based care delivery system in delivering a dementia care management program to persons with dementia and their caregivers.

Design. Randomized pragmatic clinical trial enrolling dyads of persons with dementia and their caregiver. Study participants were randomly assigned to the dementia care management program and usual care in a 2:1 ratio.

Setting and participants. The study was conducted from 2 hub sites: the University of California, San Francisco, and the University of Nebraska Medical Center in Omaha. Each hub-site team served persons with dementia and their caregivers in California, Nebraska, and Iowa in both urban and rural areas. Participants were recruited through referral by treating providers or self-referral in response to advertising presented through a community outreach event, in the news, or on the internet. Eligibility requirements included: having a dementia diagnosis made by a treating provider; age older than 45 years; Medicare or Medicaid enrollment or eligibility; presence of a caregiver willing to enroll in the study; fluency in English, Spanish, or Cantonese; and residence in California, Nebraska, or Iowa. Exclusion criteria included residence in a nursing home. Out of 2585 referred dyads of persons with dementia and caregivers, 780 met inclusion criteria and were enrolled. A 2:1 randomization yielded 512 dyads in the intervention group and 268 dyads in the control group.

Intervention. The dementia care management program was implemented through the Care Ecosystem, a telephone- and internet-based supportive care intervention delivered by care team navigators. The navigators were unlicensed but trained dementia care guides working under the supervision of an advanced practice nurse, social worker, and pharmacist. The intervention consisted of telephone calls, monthly or at a frequency determined by needs and preferences, placed by navigators over a 12-month period; the content of the calls included response to immediate needs of persons with dementia and their caregiver, screening for common problems, and provision of support and education using care plan protocols. Caregivers and persons with dementia were encouraged to initiate contact through email, mail, or telephone for dementia-related questions. Additional support was provided by an advanced practice nurse, social worker, or pharmacist, as needed, and these health care professionals conducted further communication with the persons with dementia, caregiver, or outside professionals, such as physicians, for the persons with dementia, as needed. The average number of telephone calls over the 12-month period was 15.3 (standard deviation, 11.3). Participants assigned to usual care were offered contact information on dementia and aging-related organizations, including the Alzheimer’s Association and the Area Agencies on Aging, and also were sent a quarterly newsletter with general information about dementia.

Main outcome measures. The primary outcome measure was the Quality of Life in Alzheimer’s Disease score obtained by caregiver interview. This quality of life measure includes the following aspects, each rated on an ordinal scale of 1 to 4: physical health, energy level, mood, living situation, memory, family, closest relationship, friends, self, ability to do things for fun, finances, and life as a whole. The scores range from 13 to 52, with a higher score indicating better quality of life for persons with dementia. Other outcomes included frequency of emergency room visits, hospital use, and ambulance use; caregiver depression score from the Patient Health Questionnaire scale; caregiver burden score using the 12-item Zarit Burden Interview; caregiver self-efficacy; and caregiver satisfaction.

Main results. The study found that the quality of life for persons with dementia declined more in the usual care group than in the intervention group during the 12-month study period (difference of 0.53; 95% confidence interval, 0.25-1.3; P = 0.04). Persons with dementia also had fewer emergency room visits, with a number needed to treat to prevent 1 emergency room visit of 5. The intervention did not reduce ambulance use or hospital use. Caregivers in the intervention group had a greater decline in depression when compared to usual care; the frequency of moderate to severe depression decreased from 13.4% at baseline to 7.9% at 12 months (P = 0.004). Caregiver burden declined more in the intervention group than in the control group at 12 months (P = 0.046). In terms of caregiver satisfaction, 97% of caregivers surveyed in the intervention group said they would recommend the intervention to another caregiver; 45% indicated they were very satisfied, and 33% that they were satisfied.

Conclusion. Delivering dementia care via telephone and internet through a collaborative program with care navigators can improve caregiver burden and well-being and improve quality of life, emergency room utilization, and depression for persons with dementia. In addition, the program was well received.

Commentary

Dementia, including Alzheimer’s disease, primarily affects older adults and is characterized by declines in memory and cognitive function. It is often accompanied by neuropsychological symptoms such as agitation, wandering, and physical and verbal outbursts, which are debilitating for persons living with dementia and difficult to cope with for caregivers.¹ These symptoms are often the source of caregiver stress, potentially leading to caregiver depression and eventual need for long-term institution-based care, such as nursing home placement.²

Prior literature has established the potential effect of support in improving caregiver outcomes, including caregiver stress and burden, through interventions such as enhancing resources for caregivers, teaching coping strategies to caregivers, and teaching caregivers how to manage support for their loved ones.^3,4 However, wider adoption of these interventions may be limited if the interventions involve in-person meetings or activities that take caregivers away from caregiving; the scalability of these programs is also limited by their ability to reach persons with dementia and their caregivers. These barriers are particularly important for older adults living in rural areas, where the availability of resources and distance from access to quality care may be particularly limiting.⁵ Leveraging advances in technology and telecommunication, this study examined the effects of providing dementia care support via telephone and internet using a trained, unlicensed care navigator as the main point of contact. The results showed improved quality of life for persons with dementia, reduced need for emergency room visits, and reduced caregiver burden and depression. The intervention is promising as a scalable intervention that may impact dementia care nationwide.

Despite the promising results, there are several issues regarding the intervention’s applicability and impact that future studies may help to further clarify. Although the improvement in quality of life in persons with dementia is important to document, it is unclear whether this difference is clinically significant. Also, it may be important to examine whether the 12-month program has sustained impact beyond the study period, although the intervention could be conceived as a long-term care solution. If the intervention is sustained beyond 12 months, future studies may look at other clinical outcomes, such as incidence of institutionalization and perhaps time to institutionalization. The study population consisted of persons with dementia of various stages, half of whom had mild disease. Future studies may further clarify at which stage of dementia the intervention is most useful. Other changes that occurred during the study period, such as change in the use of paid home-based support services and referrals to other relevant evaluations and treatment, may provide further clues about how the dementia care intervention achieved its beneficial effects.

Applications for Clinical Practice

From the health systems perspective, dementia care accounts for significant resources, and these costs are expected to grow as the population ages and dementia prevalence increases. Identifying potentially scalable interventions that yield clinical benefits and are sustainable from a cost perspective is an important step forward in improving care for persons with dementia and their caregivers across the nation. The use of centralized hubs to deliver this intervention and the novel use of telecommunications advances make this intervention applicable across large areas. Policy makers should explore how an intervention such as this could be established and sustained in our health care system.

–William W. Hung, MD, MPH

Study Overview

Objective. To evaluate the effect of changing from tenofovir disoproxil fumarate (TDF) –containing antiretroviral therapy (ART) to tenofovir alafenamide (TAF) –containing ART in patients ages 60 years and older living with HIV.

Design. Prospective, open-label, multicenter, randomized controlled trial.

Setting and participants. The study was completed across 36 European centers over 48 weeks. Patients were enrolled from December 12, 2015, to March 21, 2018, and were eligible to participate if they were diagnosed with HIV-1; virologically suppressed to < 50 copies/mL; on a TDF-containing ART regimen; and ≥ 60 years of age.

Intervention. Participants (n = 167) were randomly assigned in a 2:1 ratio to ART with TAF (10 mg), elvitegravir (EVG; 150 mg), cobicistat (COB; 150 mg), and emtricitabine (FTC; 200 mg) or to continued therapy with a TDF-containing ART regimen (300 mg TDF).

Main outcome measures. Primary outcome measures were the change in spine and hip bone mineral density from baseline at week 48. Secondary outcome measures included bone mineral density changes from baseline at week 24, HIV viral suppression and change in CD4 count at weeks 24 and 48, and the assessment of safety and tolerability of each ART regimen until week 48.

Main results. At 48 weeks, patients (n = 111) in the TAF+EVG+COB+FTC group had a mean 2.24% (SD, 3.27) increase in spine bone mineral density, while those in the TDF-containing group (n = 56) had a mean 0.10% decrease (SD, 3.39), a difference of 2.43% (95% confidence interval [CI], 1.34-3.52; P < 0.0001). In addition, at 48 weeks patients in the TAF+EVG+COB+FTC group had a mean 1.33% increase (SD, 2.20) in hip bone mineral density, as compared with a mean 0.73% decrease (SD, 3.21) in the TDF-containing group, a difference of 2.04% (95% CI, 1.17-2.90; P < 0.0001).

Similar results were seen in spine and hip bone mineral density in the TAF+EVG+COB+FTC group at week 24, with increases of 1.75% (P = 0.00080) and 1.35% (P = 0.00040), respectively. Both treatment groups maintained high virologic suppression. The TAF+EVG+COB+FTC group maintained 94.5% virologic suppression at week 24 and 93.6% at week 48, as compared with virologic suppression of 100% and 94.5% at weeks 24 and 48, respectively, in the TDF-containing group. However, the TAF+EVG+COB+FTC group had an increase in CD4 count from baseline (56 cells/µL), with no real change in the TDF-containing group (–1 cell/µL). Patients in the TAF+EVG+COB+FTC group had a mean 27.8 mg/g decrease in urine albumin-to-creatinine ratio (UACR) versus a 7.7 mg/g decrease in the TDF-containing group (P = 0.0042). In addition, patients in the TAF+EVG+COB+FTC group had a mean 49.8 mg/g decrease in urine protein-to-creatinine ratio (UPCR) versus a 3.8 mg/g decrease in the TDF-containing group (P = 0.0042).

Conclusion. Patients 60 years of age or older living with virologically suppressed HIV may benefit from improved bone mineral density by switching from a TDF-containing ART regimen to a TAF-containing regimen after 48 weeks, which, in turn, may help to reduce the risk for osteoporosis. Patients who were switched to a TAF-containing regimen also had favorable improvements in UACR and UPCR, which could indicate better renal function.

Commentary

The Centers for Disease Control and Prevention estimated that in 2018 nearly half of those living with HIV in the United States were older than 50 years.¹ Today, the life expectancy of patients living with HIV on ART in developed countries is similar to that of patients not living with HIV. A meta-analysis published in 2017 estimated that patients diagnosed with HIV at age 20 beginning ART have a life expectancy of 63 years, and another study estimated that life expectancy in such patients is 89.1% of that of the general population in Canada.^2,3 Overall, most people living with HIV infection are aging and at risk for medical conditions similar to persons without HIV disease. However, rates of osteoporosis in elderly patients with HIV are estimated to be 3 times greater than rates in persons without HIV.⁴ As a result, it is becoming increasingly important to find ways to decrease the risk of osteoporosis in these patients.

ART typically includes a nucleoside reverse transcriptase inhibitor (NRTI) combination and a third agent, such as an integrase strand inhibitor. Tenofovir is a commonly used backbone NRTI that comes in 2 forms, TDF (tenofovir disoproxil fumarate) and TAF (tenofovir alafenamide). Both are prodrugs that are converted to tenofovir diphosphate. TDF specifically is associated with an increased risk of bone loss and nephrotoxicity. The loss in bone mineral density is most similar to the bone loss seen with oral glucocorticoids.⁵ TDF has been shown to increase plasma levels of RANKL and tumor necrosis factor-α, leading to increased bone resorption.⁶ The long-term effects of TDF- versus TAF-containing ART on bone mineral density have, to our knowledge, not been compared previously in a randomized control study. The significance of demonstrating an increase in bone mineral density in the prevention of osteoporotic bone fracture in people living with HIV is less clear. A long-term cohort study completed in Japan looking at patients on TDF showed an increased risk of bone fractures in both older postmenopausal women and younger men.⁷ However, a retrospective cohort study looking at 1981 patients with HIV found no association between bone fractures and TDF.⁸

This randomized controlled trial used appropriate methods to measure the reported primary and secondary endpoints; however, it would be of benefit to continue following these patients to measure their true long-term risk of osteoporosis-related complications. In terms of the study’s secondary endpoints, it is notable that the patients maintained HIV viral suppression after the switch and CD4 counts remained stable (with a slight increase observed in the TAF-containing ART cohort).

In regard to the patient’s renal function, patients in the TAF group had significantly improved UACR and UPCR, which likely reflects improved glomerular filtration. Improved renal function is also increasingly important for patients with HIV, as up to 48.5% have some form of chronic kidney disease.⁹

This study shows that making the switch from TDF- to TAF-containing ART can lead to improved bone mineral density. We can extrapolate that switching may lead to a decreased risk of osteoporosis and osteoporosis-related complications, such as bone fracture, but this needs to be investigated in more detail. As demonstrated in this study, switching from a TDF- to a TAF-containing regimen can also lead to improved renal function while maintaining HIV viral suppression and CD4 counts.

Unfortunately, the regimen selected with TAF in this study (elvitegravir, cobicistat, and emtricitabine) includes cobicistat, which is no longer recommended as initial therapy due to its risk of drug-drug interactions, and elvitegravir, which has a lower barrier to resistance than other integrase strand inhibitors.^10,11 The United States Department of Health and Human Services guidelines and the International Antiviral Society-USA Panel suggest using several other TAF-containing regimens for beginning or even switching therapy in older patients.^10,11

When choosing between either a TAF- or a TDF-containing regimen to treat HIV infection in older patients, increasing evidence shows that using a TAF-containing ART regimen may be more beneficial for people living and aging with virologically suppressed HIV infection.

–Sean P. Bliven, and Norman L. Beatty, MD, University of Florida College of Medicine, Division of Infectious Diseases and Global Medicine, Gainesville, FL

Study Overview

Objective. To evaluate the effect of changing from tenofovir disoproxil fumarate (TDF) –containing antiretroviral therapy (ART) to tenofovir alafenamide (TAF) –containing ART in patients ages 60 years and older living with HIV.

Design. Prospective, open-label, multicenter, randomized controlled trial.

Setting and participants. The study was completed across 36 European centers over 48 weeks. Patients were enrolled from December 12, 2015, to March 21, 2018, and were eligible to participate if they were diagnosed with HIV-1; virologically suppressed to < 50 copies/mL; on a TDF-containing ART regimen; and ≥ 60 years of age.

Intervention. Participants (n = 167) were randomly assigned in a 2:1 ratio to ART with TAF (10 mg), elvitegravir (EVG; 150 mg), cobicistat (COB; 150 mg), and emtricitabine (FTC; 200 mg) or to continued therapy with a TDF-containing ART regimen (300 mg TDF).

Main outcome measures. Primary outcome measures were the change in spine and hip bone mineral density from baseline at week 48. Secondary outcome measures included bone mineral density changes from baseline at week 24, HIV viral suppression and change in CD4 count at weeks 24 and 48, and the assessment of safety and tolerability of each ART regimen until week 48.

Main results. At 48 weeks, patients (n = 111) in the TAF+EVG+COB+FTC group had a mean 2.24% (SD, 3.27) increase in spine bone mineral density, while those in the TDF-containing group (n = 56) had a mean 0.10% decrease (SD, 3.39), a difference of 2.43% (95% confidence interval [CI], 1.34-3.52; P < 0.0001). In addition, at 48 weeks patients in the TAF+EVG+COB+FTC group had a mean 1.33% increase (SD, 2.20) in hip bone mineral density, as compared with a mean 0.73% decrease (SD, 3.21) in the TDF-containing group, a difference of 2.04% (95% CI, 1.17-2.90; P < 0.0001).

Similar results were seen in spine and hip bone mineral density in the TAF+EVG+COB+FTC group at week 24, with increases of 1.75% (P = 0.00080) and 1.35% (P = 0.00040), respectively. Both treatment groups maintained high virologic suppression. The TAF+EVG+COB+FTC group maintained 94.5% virologic suppression at week 24 and 93.6% at week 48, as compared with virologic suppression of 100% and 94.5% at weeks 24 and 48, respectively, in the TDF-containing group. However, the TAF+EVG+COB+FTC group had an increase in CD4 count from baseline (56 cells/µL), with no real change in the TDF-containing group (–1 cell/µL). Patients in the TAF+EVG+COB+FTC group had a mean 27.8 mg/g decrease in urine albumin-to-creatinine ratio (UACR) versus a 7.7 mg/g decrease in the TDF-containing group (P = 0.0042). In addition, patients in the TAF+EVG+COB+FTC group had a mean 49.8 mg/g decrease in urine protein-to-creatinine ratio (UPCR) versus a 3.8 mg/g decrease in the TDF-containing group (P = 0.0042).

Conclusion. Patients 60 years of age or older living with virologically suppressed HIV may benefit from improved bone mineral density by switching from a TDF-containing ART regimen to a TAF-containing regimen after 48 weeks, which, in turn, may help to reduce the risk for osteoporosis. Patients who were switched to a TAF-containing regimen also had favorable improvements in UACR and UPCR, which could indicate better renal function.

Commentary

The Centers for Disease Control and Prevention estimated that in 2018 nearly half of those living with HIV in the United States were older than 50 years.¹ Today, the life expectancy of patients living with HIV on ART in developed countries is similar to that of patients not living with HIV. A meta-analysis published in 2017 estimated that patients diagnosed with HIV at age 20 beginning ART have a life expectancy of 63 years, and another study estimated that life expectancy in such patients is 89.1% of that of the general population in Canada.^2,3 Overall, most people living with HIV infection are aging and at risk for medical conditions similar to persons without HIV disease. However, rates of osteoporosis in elderly patients with HIV are estimated to be 3 times greater than rates in persons without HIV.⁴ As a result, it is becoming increasingly important to find ways to decrease the risk of osteoporosis in these patients.

ART typically includes a nucleoside reverse transcriptase inhibitor (NRTI) combination and a third agent, such as an integrase strand inhibitor. Tenofovir is a commonly used backbone NRTI that comes in 2 forms, TDF (tenofovir disoproxil fumarate) and TAF (tenofovir alafenamide). Both are prodrugs that are converted to tenofovir diphosphate. TDF specifically is associated with an increased risk of bone loss and nephrotoxicity. The loss in bone mineral density is most similar to the bone loss seen with oral glucocorticoids.⁵ TDF has been shown to increase plasma levels of RANKL and tumor necrosis factor-α, leading to increased bone resorption.⁶ The long-term effects of TDF- versus TAF-containing ART on bone mineral density have, to our knowledge, not been compared previously in a randomized control study. The significance of demonstrating an increase in bone mineral density in the prevention of osteoporotic bone fracture in people living with HIV is less clear. A long-term cohort study completed in Japan looking at patients on TDF showed an increased risk of bone fractures in both older postmenopausal women and younger men.⁷ However, a retrospective cohort study looking at 1981 patients with HIV found no association between bone fractures and TDF.⁸

This randomized controlled trial used appropriate methods to measure the reported primary and secondary endpoints; however, it would be of benefit to continue following these patients to measure their true long-term risk of osteoporosis-related complications. In terms of the study’s secondary endpoints, it is notable that the patients maintained HIV viral suppression after the switch and CD4 counts remained stable (with a slight increase observed in the TAF-containing ART cohort).

In regard to the patient’s renal function, patients in the TAF group had significantly improved UACR and UPCR, which likely reflects improved glomerular filtration. Improved renal function is also increasingly important for patients with HIV, as up to 48.5% have some form of chronic kidney disease.⁹

This study shows that making the switch from TDF- to TAF-containing ART can lead to improved bone mineral density. We can extrapolate that switching may lead to a decreased risk of osteoporosis and osteoporosis-related complications, such as bone fracture, but this needs to be investigated in more detail. As demonstrated in this study, switching from a TDF- to a TAF-containing regimen can also lead to improved renal function while maintaining HIV viral suppression and CD4 counts.

Unfortunately, the regimen selected with TAF in this study (elvitegravir, cobicistat, and emtricitabine) includes cobicistat, which is no longer recommended as initial therapy due to its risk of drug-drug interactions, and elvitegravir, which has a lower barrier to resistance than other integrase strand inhibitors.^10,11 The United States Department of Health and Human Services guidelines and the International Antiviral Society-USA Panel suggest using several other TAF-containing regimens for beginning or even switching therapy in older patients.^10,11

When choosing between either a TAF- or a TDF-containing regimen to treat HIV infection in older patients, increasing evidence shows that using a TAF-containing ART regimen may be more beneficial for people living and aging with virologically suppressed HIV infection.

–Sean P. Bliven, and Norman L. Beatty, MD, University of Florida College of Medicine, Division of Infectious Diseases and Global Medicine, Gainesville, FL

Study Overview

Objective. To evaluate the effect of changing from tenofovir disoproxil fumarate (TDF) –containing antiretroviral therapy (ART) to tenofovir alafenamide (TAF) –containing ART in patients ages 60 years and older living with HIV.

Design. Prospective, open-label, multicenter, randomized controlled trial.

Setting and participants. The study was completed across 36 European centers over 48 weeks. Patients were enrolled from December 12, 2015, to March 21, 2018, and were eligible to participate if they were diagnosed with HIV-1; virologically suppressed to < 50 copies/mL; on a TDF-containing ART regimen; and ≥ 60 years of age.

Intervention. Participants (n = 167) were randomly assigned in a 2:1 ratio to ART with TAF (10 mg), elvitegravir (EVG; 150 mg), cobicistat (COB; 150 mg), and emtricitabine (FTC; 200 mg) or to continued therapy with a TDF-containing ART regimen (300 mg TDF).

Main outcome measures. Primary outcome measures were the change in spine and hip bone mineral density from baseline at week 48. Secondary outcome measures included bone mineral density changes from baseline at week 24, HIV viral suppression and change in CD4 count at weeks 24 and 48, and the assessment of safety and tolerability of each ART regimen until week 48.

Main results. At 48 weeks, patients (n = 111) in the TAF+EVG+COB+FTC group had a mean 2.24% (SD, 3.27) increase in spine bone mineral density, while those in the TDF-containing group (n = 56) had a mean 0.10% decrease (SD, 3.39), a difference of 2.43% (95% confidence interval [CI], 1.34-3.52; P < 0.0001). In addition, at 48 weeks patients in the TAF+EVG+COB+FTC group had a mean 1.33% increase (SD, 2.20) in hip bone mineral density, as compared with a mean 0.73% decrease (SD, 3.21) in the TDF-containing group, a difference of 2.04% (95% CI, 1.17-2.90; P < 0.0001).

Similar results were seen in spine and hip bone mineral density in the TAF+EVG+COB+FTC group at week 24, with increases of 1.75% (P = 0.00080) and 1.35% (P = 0.00040), respectively. Both treatment groups maintained high virologic suppression. The TAF+EVG+COB+FTC group maintained 94.5% virologic suppression at week 24 and 93.6% at week 48, as compared with virologic suppression of 100% and 94.5% at weeks 24 and 48, respectively, in the TDF-containing group. However, the TAF+EVG+COB+FTC group had an increase in CD4 count from baseline (56 cells/µL), with no real change in the TDF-containing group (–1 cell/µL). Patients in the TAF+EVG+COB+FTC group had a mean 27.8 mg/g decrease in urine albumin-to-creatinine ratio (UACR) versus a 7.7 mg/g decrease in the TDF-containing group (P = 0.0042). In addition, patients in the TAF+EVG+COB+FTC group had a mean 49.8 mg/g decrease in urine protein-to-creatinine ratio (UPCR) versus a 3.8 mg/g decrease in the TDF-containing group (P = 0.0042).

Conclusion. Patients 60 years of age or older living with virologically suppressed HIV may benefit from improved bone mineral density by switching from a TDF-containing ART regimen to a TAF-containing regimen after 48 weeks, which, in turn, may help to reduce the risk for osteoporosis. Patients who were switched to a TAF-containing regimen also had favorable improvements in UACR and UPCR, which could indicate better renal function.

Commentary

The Centers for Disease Control and Prevention estimated that in 2018 nearly half of those living with HIV in the United States were older than 50 years.¹ Today, the life expectancy of patients living with HIV on ART in developed countries is similar to that of patients not living with HIV. A meta-analysis published in 2017 estimated that patients diagnosed with HIV at age 20 beginning ART have a life expectancy of 63 years, and another study estimated that life expectancy in such patients is 89.1% of that of the general population in Canada.^2,3 Overall, most people living with HIV infection are aging and at risk for medical conditions similar to persons without HIV disease. However, rates of osteoporosis in elderly patients with HIV are estimated to be 3 times greater than rates in persons without HIV.⁴ As a result, it is becoming increasingly important to find ways to decrease the risk of osteoporosis in these patients.

ART typically includes a nucleoside reverse transcriptase inhibitor (NRTI) combination and a third agent, such as an integrase strand inhibitor. Tenofovir is a commonly used backbone NRTI that comes in 2 forms, TDF (tenofovir disoproxil fumarate) and TAF (tenofovir alafenamide). Both are prodrugs that are converted to tenofovir diphosphate. TDF specifically is associated with an increased risk of bone loss and nephrotoxicity. The loss in bone mineral density is most similar to the bone loss seen with oral glucocorticoids.⁵ TDF has been shown to increase plasma levels of RANKL and tumor necrosis factor-α, leading to increased bone resorption.⁶ The long-term effects of TDF- versus TAF-containing ART on bone mineral density have, to our knowledge, not been compared previously in a randomized control study. The significance of demonstrating an increase in bone mineral density in the prevention of osteoporotic bone fracture in people living with HIV is less clear. A long-term cohort study completed in Japan looking at patients on TDF showed an increased risk of bone fractures in both older postmenopausal women and younger men.⁷ However, a retrospective cohort study looking at 1981 patients with HIV found no association between bone fractures and TDF.⁸

This randomized controlled trial used appropriate methods to measure the reported primary and secondary endpoints; however, it would be of benefit to continue following these patients to measure their true long-term risk of osteoporosis-related complications. In terms of the study’s secondary endpoints, it is notable that the patients maintained HIV viral suppression after the switch and CD4 counts remained stable (with a slight increase observed in the TAF-containing ART cohort).

In regard to the patient’s renal function, patients in the TAF group had significantly improved UACR and UPCR, which likely reflects improved glomerular filtration. Improved renal function is also increasingly important for patients with HIV, as up to 48.5% have some form of chronic kidney disease.⁹

This study shows that making the switch from TDF- to TAF-containing ART can lead to improved bone mineral density. We can extrapolate that switching may lead to a decreased risk of osteoporosis and osteoporosis-related complications, such as bone fracture, but this needs to be investigated in more detail. As demonstrated in this study, switching from a TDF- to a TAF-containing regimen can also lead to improved renal function while maintaining HIV viral suppression and CD4 counts.

Unfortunately, the regimen selected with TAF in this study (elvitegravir, cobicistat, and emtricitabine) includes cobicistat, which is no longer recommended as initial therapy due to its risk of drug-drug interactions, and elvitegravir, which has a lower barrier to resistance than other integrase strand inhibitors.^10,11 The United States Department of Health and Human Services guidelines and the International Antiviral Society-USA Panel suggest using several other TAF-containing regimens for beginning or even switching therapy in older patients.^10,11

When choosing between either a TAF- or a TDF-containing regimen to treat HIV infection in older patients, increasing evidence shows that using a TAF-containing ART regimen may be more beneficial for people living and aging with virologically suppressed HIV infection.

–Sean P. Bliven, and Norman L. Beatty, MD, University of Florida College of Medicine, Division of Infectious Diseases and Global Medicine, Gainesville, FL

Study Overview

Objective. To determine whether percutaneous coronary intervention (PCI) of a nonculprit lesion in patients with ST-segment elevation myocardial infarction (STEMI) reduces the risk of cardiovascular death or myocardial infarction.

Design. International, multicenter, randomized controlled trial blinded to outcome.

Setting and participants. Patients with STEMI who had multivessel coronary disease and had undergone successful PCI to the culprit lesion.

Intervention. A total of 4041 patients were randomly assigned to either PCI of angiographically significant nonculprit lesions or optimal medical therapy without further revascularization. Randomization was stratified according to intended timing of nonculprit lesion PCI (either during or after the index hospitalization).

Main outcome measures. The first co-primary endpoint was the composite of cardiovascular death or myocardial infarction (MI). The second co-primary endpoint was the composite of cardiovascular death, MI or ischemia-driven revascularization.

Main results. At a median follow-up of 3 years, the composite of cardiovascular death or MI occurred in 158 of the 2016 patients (7.8%) in the nonculprit PCI group and in 213 of the 2025 patients (10.5%) in the culprit-lesion-only group (hazard ratio, 0.73; 95% confidence interval [CI], 0.60-0.91; P = 0.004). The second co-primary endpoint occurred in 179 patients (8.9%) in the nonculprit PCI group and in 339 patients (16.7%) in the culprit-lesion-only group (hazard ratio, 0.51; 95% CI, 0.43-0.61; P < 0.001).

Conclusion. Among patients with STEMI and multivessel disease, those who underwent complete revascularization with nonculprit lesion PCI had lower rates of cardiovascular death or MI compared to patients with culprit-lesion-only revascularization.

Patients presenting with STEMI often have multivessel disease.¹ Although it is known that mortality can be reduced by early revascularization of the culprit vessel,² whether the nonculprit vessel should be revascularized at the time of presentation with STEMI remains controversial.

Recently, multiple studies have reported the benefit of nonculprit vessel revascularization in patients presenting with hemodynamically stable STEMI. Four trials (PRAMI, CvPRIT, DANAMI-PRIMULTI, and COMPARE ACUTE) investigated this clinical question with different designs, and all reported benefit of nonculprit vessel revascularization compared to a culprit-only strategy.^3-6 However, the differences in the composite endpoints were mainly driven by the softer endpoints used in these trials, such as refractory angina and ischemia-driven revascularization, and none of these previous trials had adequate power to evaluate differences in hard outcomes, such as death or MI.

In this context, Mehta et al investigated whether achieving complete revascularization by performing PCI on nonculprit vessels would improve the composite of cardiovascular death or MI compared to the culprit-only strategy by conducting a well-designed randomized controlled study. At median follow-up of 3 years, patients who underwent nonculprit vessel PCI had a lower incidence of death or MI compared to those who received the culprit-only strategy (7.8% versus 10.5%). The second co-primary endpoint (composite of death, MI, or ischemia-driven revascularization) also occurred significantly less frequently in the nonculprit PCI group than in the culprit-only PCI group (8.9% versus 16.7%).

The current study has a number of strengths. First, this was a multicenter, international study, and a large number of patients were enrolled (> 4000), achieving adequate power to evaluate for the composite of death and MI. Second, the treatments the patients received reflect contemporary medical therapy and interventional practice: the third-generation thienopyridine ticagrelor, high-dose statins, and ACE inhibitors were prescribed at high rates, and radial access (> 80%) and current-generation drug-eluting stents were used at high rates as well. Third, all angiograms were reviewed by the core lab to evaluate for completeness of revascularization. Fourth, the trial mandated use of fractional flow reserve to assess lesion stenosis 50% to 69% before considering revascularization, ensuring that only ischemic or very-high-grade lesions were revascularized. Fifth, the crossover rate in each group was low compared to the previous studies (4.7% into the complete revascularization group, 3.9% into the lesion-only group). Finally, this study evaluated the timing of the nonculprit PCI. Randomization to each group was stratified according to the intended timing of the nonculprit PCI during the index hospitalization or after hospital discharge (within 45 days). They found that benefit was consistent regardless of when the nonculprit PCI was performed.

Although the COMPLETE study’s design has a number of strengths, it is important to note that patients enrolled in this trial represent a lower-risk STEMI population. Patients with complex anatomy likely were not included, as evidenced by a lower SYNTAX score (mean, 16). Furthermore, no patients who presented with STEMI complicated by cardiogenic shock were enrolled. In the recent CULPRIT SHOCK trial, which focused on patients who had multivessel disease, acute MI, and cardiogenic shock, patients who underwent the culprit-only strategy had a lower rate of death or renal replacement therapy, as compared to patients who underwent immediate complete revascularization.⁷ Therefore, whether the findings from the COMPLETE study can be extended to a sicker population requires further study.

In 2015, the results from the previous trials, such as PRAMI and CvPRIT, led to a focused update of US PCI guidelines.⁸ Recommendations for noninfarct-related artery PCI in hemodynamically stable patients presenting with acute MI were upgraded from class III to class IIb. The results from the COMPLETE trial will likely influence the future guidelines, with stronger recommendations toward complete revascularization in patients presenting with hemodynamically stable STEMI.

Applications for Clinical Practice

In patients presenting with hemodynamically stable STEMI, staged complete revascularization, including the nonculprit vessel, should be considered.

– Taishi Hirai, MD, University of Missouri, Columbia, MO, and John EA Blair, MD, University of Chicago Medical Center, Chicago, IL

Study Overview

Objective. To determine whether percutaneous coronary intervention (PCI) of a nonculprit lesion in patients with ST-segment elevation myocardial infarction (STEMI) reduces the risk of cardiovascular death or myocardial infarction.

Design. International, multicenter, randomized controlled trial blinded to outcome.

Setting and participants. Patients with STEMI who had multivessel coronary disease and had undergone successful PCI to the culprit lesion.

Intervention. A total of 4041 patients were randomly assigned to either PCI of angiographically significant nonculprit lesions or optimal medical therapy without further revascularization. Randomization was stratified according to intended timing of nonculprit lesion PCI (either during or after the index hospitalization).

Main outcome measures. The first co-primary endpoint was the composite of cardiovascular death or myocardial infarction (MI). The second co-primary endpoint was the composite of cardiovascular death, MI or ischemia-driven revascularization.

Main results. At a median follow-up of 3 years, the composite of cardiovascular death or MI occurred in 158 of the 2016 patients (7.8%) in the nonculprit PCI group and in 213 of the 2025 patients (10.5%) in the culprit-lesion-only group (hazard ratio, 0.73; 95% confidence interval [CI], 0.60-0.91; P = 0.004). The second co-primary endpoint occurred in 179 patients (8.9%) in the nonculprit PCI group and in 339 patients (16.7%) in the culprit-lesion-only group (hazard ratio, 0.51; 95% CI, 0.43-0.61; P < 0.001).

Conclusion. Among patients with STEMI and multivessel disease, those who underwent complete revascularization with nonculprit lesion PCI had lower rates of cardiovascular death or MI compared to patients with culprit-lesion-only revascularization.

Patients presenting with STEMI often have multivessel disease.¹ Although it is known that mortality can be reduced by early revascularization of the culprit vessel,² whether the nonculprit vessel should be revascularized at the time of presentation with STEMI remains controversial.

Recently, multiple studies have reported the benefit of nonculprit vessel revascularization in patients presenting with hemodynamically stable STEMI. Four trials (PRAMI, CvPRIT, DANAMI-PRIMULTI, and COMPARE ACUTE) investigated this clinical question with different designs, and all reported benefit of nonculprit vessel revascularization compared to a culprit-only strategy.^3-6 However, the differences in the composite endpoints were mainly driven by the softer endpoints used in these trials, such as refractory angina and ischemia-driven revascularization, and none of these previous trials had adequate power to evaluate differences in hard outcomes, such as death or MI.

In this context, Mehta et al investigated whether achieving complete revascularization by performing PCI on nonculprit vessels would improve the composite of cardiovascular death or MI compared to the culprit-only strategy by conducting a well-designed randomized controlled study. At median follow-up of 3 years, patients who underwent nonculprit vessel PCI had a lower incidence of death or MI compared to those who received the culprit-only strategy (7.8% versus 10.5%). The second co-primary endpoint (composite of death, MI, or ischemia-driven revascularization) also occurred significantly less frequently in the nonculprit PCI group than in the culprit-only PCI group (8.9% versus 16.7%).

The current study has a number of strengths. First, this was a multicenter, international study, and a large number of patients were enrolled (> 4000), achieving adequate power to evaluate for the composite of death and MI. Second, the treatments the patients received reflect contemporary medical therapy and interventional practice: the third-generation thienopyridine ticagrelor, high-dose statins, and ACE inhibitors were prescribed at high rates, and radial access (> 80%) and current-generation drug-eluting stents were used at high rates as well. Third, all angiograms were reviewed by the core lab to evaluate for completeness of revascularization. Fourth, the trial mandated use of fractional flow reserve to assess lesion stenosis 50% to 69% before considering revascularization, ensuring that only ischemic or very-high-grade lesions were revascularized. Fifth, the crossover rate in each group was low compared to the previous studies (4.7% into the complete revascularization group, 3.9% into the lesion-only group). Finally, this study evaluated the timing of the nonculprit PCI. Randomization to each group was stratified according to the intended timing of the nonculprit PCI during the index hospitalization or after hospital discharge (within 45 days). They found that benefit was consistent regardless of when the nonculprit PCI was performed.

Although the COMPLETE study’s design has a number of strengths, it is important to note that patients enrolled in this trial represent a lower-risk STEMI population. Patients with complex anatomy likely were not included, as evidenced by a lower SYNTAX score (mean, 16). Furthermore, no patients who presented with STEMI complicated by cardiogenic shock were enrolled. In the recent CULPRIT SHOCK trial, which focused on patients who had multivessel disease, acute MI, and cardiogenic shock, patients who underwent the culprit-only strategy had a lower rate of death or renal replacement therapy, as compared to patients who underwent immediate complete revascularization.⁷ Therefore, whether the findings from the COMPLETE study can be extended to a sicker population requires further study.

In 2015, the results from the previous trials, such as PRAMI and CvPRIT, led to a focused update of US PCI guidelines.⁸ Recommendations for noninfarct-related artery PCI in hemodynamically stable patients presenting with acute MI were upgraded from class III to class IIb. The results from the COMPLETE trial will likely influence the future guidelines, with stronger recommendations toward complete revascularization in patients presenting with hemodynamically stable STEMI.

Applications for Clinical Practice

In patients presenting with hemodynamically stable STEMI, staged complete revascularization, including the nonculprit vessel, should be considered.

– Taishi Hirai, MD, University of Missouri, Columbia, MO, and John EA Blair, MD, University of Chicago Medical Center, Chicago, IL

Study Overview

Objective. To determine whether percutaneous coronary intervention (PCI) of a nonculprit lesion in patients with ST-segment elevation myocardial infarction (STEMI) reduces the risk of cardiovascular death or myocardial infarction.

Design. International, multicenter, randomized controlled trial blinded to outcome.

Setting and participants. Patients with STEMI who had multivessel coronary disease and had undergone successful PCI to the culprit lesion.

Intervention. A total of 4041 patients were randomly assigned to either PCI of angiographically significant nonculprit lesions or optimal medical therapy without further revascularization. Randomization was stratified according to intended timing of nonculprit lesion PCI (either during or after the index hospitalization).

Main outcome measures. The first co-primary endpoint was the composite of cardiovascular death or myocardial infarction (MI). The second co-primary endpoint was the composite of cardiovascular death, MI or ischemia-driven revascularization.

Main results. At a median follow-up of 3 years, the composite of cardiovascular death or MI occurred in 158 of the 2016 patients (7.8%) in the nonculprit PCI group and in 213 of the 2025 patients (10.5%) in the culprit-lesion-only group (hazard ratio, 0.73; 95% confidence interval [CI], 0.60-0.91; P = 0.004). The second co-primary endpoint occurred in 179 patients (8.9%) in the nonculprit PCI group and in 339 patients (16.7%) in the culprit-lesion-only group (hazard ratio, 0.51; 95% CI, 0.43-0.61; P < 0.001).

Conclusion. Among patients with STEMI and multivessel disease, those who underwent complete revascularization with nonculprit lesion PCI had lower rates of cardiovascular death or MI compared to patients with culprit-lesion-only revascularization.

Patients presenting with STEMI often have multivessel disease.¹ Although it is known that mortality can be reduced by early revascularization of the culprit vessel,² whether the nonculprit vessel should be revascularized at the time of presentation with STEMI remains controversial.

Recently, multiple studies have reported the benefit of nonculprit vessel revascularization in patients presenting with hemodynamically stable STEMI. Four trials (PRAMI, CvPRIT, DANAMI-PRIMULTI, and COMPARE ACUTE) investigated this clinical question with different designs, and all reported benefit of nonculprit vessel revascularization compared to a culprit-only strategy.^3-6 However, the differences in the composite endpoints were mainly driven by the softer endpoints used in these trials, such as refractory angina and ischemia-driven revascularization, and none of these previous trials had adequate power to evaluate differences in hard outcomes, such as death or MI.

In this context, Mehta et al investigated whether achieving complete revascularization by performing PCI on nonculprit vessels would improve the composite of cardiovascular death or MI compared to the culprit-only strategy by conducting a well-designed randomized controlled study. At median follow-up of 3 years, patients who underwent nonculprit vessel PCI had a lower incidence of death or MI compared to those who received the culprit-only strategy (7.8% versus 10.5%). The second co-primary endpoint (composite of death, MI, or ischemia-driven revascularization) also occurred significantly less frequently in the nonculprit PCI group than in the culprit-only PCI group (8.9% versus 16.7%).

The current study has a number of strengths. First, this was a multicenter, international study, and a large number of patients were enrolled (> 4000), achieving adequate power to evaluate for the composite of death and MI. Second, the treatments the patients received reflect contemporary medical therapy and interventional practice: the third-generation thienopyridine ticagrelor, high-dose statins, and ACE inhibitors were prescribed at high rates, and radial access (> 80%) and current-generation drug-eluting stents were used at high rates as well. Third, all angiograms were reviewed by the core lab to evaluate for completeness of revascularization. Fourth, the trial mandated use of fractional flow reserve to assess lesion stenosis 50% to 69% before considering revascularization, ensuring that only ischemic or very-high-grade lesions were revascularized. Fifth, the crossover rate in each group was low compared to the previous studies (4.7% into the complete revascularization group, 3.9% into the lesion-only group). Finally, this study evaluated the timing of the nonculprit PCI. Randomization to each group was stratified according to the intended timing of the nonculprit PCI during the index hospitalization or after hospital discharge (within 45 days). They found that benefit was consistent regardless of when the nonculprit PCI was performed.

Although the COMPLETE study’s design has a number of strengths, it is important to note that patients enrolled in this trial represent a lower-risk STEMI population. Patients with complex anatomy likely were not included, as evidenced by a lower SYNTAX score (mean, 16). Furthermore, no patients who presented with STEMI complicated by cardiogenic shock were enrolled. In the recent CULPRIT SHOCK trial, which focused on patients who had multivessel disease, acute MI, and cardiogenic shock, patients who underwent the culprit-only strategy had a lower rate of death or renal replacement therapy, as compared to patients who underwent immediate complete revascularization.⁷ Therefore, whether the findings from the COMPLETE study can be extended to a sicker population requires further study.

In 2015, the results from the previous trials, such as PRAMI and CvPRIT, led to a focused update of US PCI guidelines.⁸ Recommendations for noninfarct-related artery PCI in hemodynamically stable patients presenting with acute MI were upgraded from class III to class IIb. The results from the COMPLETE trial will likely influence the future guidelines, with stronger recommendations toward complete revascularization in patients presenting with hemodynamically stable STEMI.

Applications for Clinical Practice

In patients presenting with hemodynamically stable STEMI, staged complete revascularization, including the nonculprit vessel, should be considered.

– Taishi Hirai, MD, University of Missouri, Columbia, MO, and John EA Blair, MD, University of Chicago Medical Center, Chicago, IL