Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

Automated early warning systems (EWSs) use data inputs to recognize clinical states requiring time-sensitive intervention and then generate notifications through different modalities to clinicians. EWSs serve as common tools for improving the recognition and treatment of important clinical states such as sepsis. However, despite the early enthusiasm, these warning systems have often yielded disappointing outcomes. In sepsis, for example, EWSs have shown mixed results in clinical trials, and concerns regarding the overuse of EWSs in diagnosing sepsis have grown.^1-4 We argue that inattention to the importance of timing in EWS training and evaluation provides one reason that EWSs have underperformed. Thus, to improve care, a warning system must not only identify the clinical state accurately, but it must also do so in a sufficiently timely manner to implement the associated interventions, such as administration of antibiotics for sepsis. Although the literature has occasionally highlighted the importance of timing in electronic surveillance systems, no one has linked the temporal dependence of performance metrics and intervention feasibility to the failure of such warning systems and explained how to operationalize timing in their development.^5-8 Using sepsis as an example, we explain why timing is important and propose new metrics and strategies for training and evaluating EWS models. EWSs are divided into two types: detection systems that recognize critical illnesses at a particular moment and prediction systems that estimate risk of deterioration over varying time frames.⁹ We focus primarily on detection systems, but our analysis is also important for prediction systems, which we will discuss in the last section.

EWS metrics have evolved from focusing on crude measures of discrimination to more clinically relevant metrics, such as the positive predictive value (PPV). The common performance metrics, including the c-statistic, evaluate the performance of EWSs in distinguishing events from nonevents, such as the presence or absence of sepsis in hospitalized patients. However, the c-statistic does not account for disease prevalence. A given c-statistic is compatible with a wide range of PPVs; a low PPV may limit an EWS’s usefulness to promote interventions and generate increased alert fatigue.¹⁰

However, the PPV, although important, provides no information on the timing of state recognition in relation to clinical time zero. Time zero is the first moment at which a critical state can be recognized based on available data and current medical science. Different approaches, including laboratory values, clinical assessments, retrospective chart reviews, triage times, and others, have been used to measure time zero.^8,11-13 All these approaches feature advantages and disadvantages; the evaluation of timing will exhibit sensitivity to the approach used.¹⁴ Further work is needed to gain additional insights into the measurement of time zero.

a) 50% true cases of sepsis, with a mean time of 35 minutes after clinical time zero;

b) 50% true cases, with a mean time of 60 minutes before clinical time zero (prediction EWS);

c) 50% true cases of sepsis, with a mean time of 1.3 days since clinical time zero, but with 70% of these cases undiagnosed at the time of EWS detection;

d) 50% true cases of cases, with mean time of 1.3 days since clinical time zero, that is, all cases among those promptly detected and treated through routine clinician oversight.

Each of these situations features differing clinical utility to help meet the hospital objective of increasing early administration of antibiotics. More generally, three dimensions of timing are important for detection systems. The first dimension is the timing of detection relative to time zero. The second is the timing relative to ”real-world” clinician detection. The third is timing with respect to the associated clinical objective. For a given PPV, an EWS performs better when detecting a state (1) at, near, or in advance of time zero, (2) prior to clinician detection, and (3) sufficiently in advance of an operational objective to promote change. On the other hand, when an EWS consistently sends alerts after clinician action, it serves a lesser purpose and risks causing alert fatigue; such cases have been described in studies.¹⁵

Acknowledging the importance of timing features implications for researchers and health system leaders. Researchers who develop EWS should include how these systems perform relative to both time zero and critical milestones in the clinical course. Operational leadership should understand the trade-offs that occur between alert fatigue (through lower PPV at the margin with earlier detection) and lead time to implement an intervention. Navigating these trade-offs involves a complex organizational decision. The “number needed to evaluate” is one way to quantify this fatigue factor.¹⁶ Such a measure gives a sense of the number of cases a clinician will need to evaluate per event. Collaborations between clinical leadership, operational leadership, and data scientists are needed to determine how to evaluate individual systems.

A good metric should capture the three important dimensions of timing while retaining intuitiveness to clinicians and leadership. One graphical option involves plotting the PPVs over time and relative to the clinical state evolution (Figure). This PPV-over-time curve shows when true positives occur relative to the time course of sepsis, including the three major dimensions of timing. This curve can also show a “clinically important window (CIW)”, which is bounded on the right by the latest point in time when recognition could still meet the clinical objective. For sepsis, the curve might be bounded at 2.5 hours to meet an objective of antibiotics within three hours, with the assumption that 0.5 hour is needed for a response. For detection systems, the window would be bounded on the left by clinical time zero. The graph can also designate the point when most cases of sepsis have been recognized clinically with historical data. The Figure depicts an example curve for a detection model.

The metrics derived from this curve may be used alongside the PPV for training and evaluation. Often, adjusting the PPV for its relationship to time zero and the CIW will aid in recognizing the existence of a time beyond which detection fails to help achieve the intended intervention. Detection beyond the window should not credited as a true positive if it fails to facilitate the objective. One option is to credit detection at or before time zero as one and discount later detection by the delay from time zero. More specifically, a true positive could be discounted by the difference between the end of the CIW and the moment of detection divided by the CIW length. This discounted PPV could be displayed alongside the PPV to gauge the temporal dimension of performance and be used for training.

The use of timing places additional demands on validation owing to the need for a time-based gold standard. In such a case, the unit of analysis in system development might not be the patient encounter but rather the patient-hour or patient-15-minute epoch, depending on how frequently the EWS updates risk information and may alert. By contrast, the sepsis detection models used in administrative databases rely on an encounter-level PPV, which provides more limited information compared with real-time EWSs.¹⁷ When time zero cannot be measured, alternatives may be used to capture several dimensions of timing; these alternatives include measurement of the percentage of cases that recognize the event prior to clinicians.¹⁵

Detection systems face the limitation that they lack the capability to identify a state before its occurrence. Prediction systems are more likely to be actionable, as they provide more lead time for intervention, but accurate prediction models are also more difficult to develop. With a predictive system, an additional dimension of timing becomes important: the time horizon for prediction. Prediction models may be trained to recognize a state within a specific time frame (eg, 6, 12, or 24 hours), and test characteristics, including PPV, may vary with the window.¹⁸ A given PPV (of eventual development of sepsis) is compatible with varying time windows and thus again lacks important information on performance.

The timing relative to clinical time zero remains important for prediction. For a predictive EWS, the graph in the figure may be expected to shift to the left. Models with good performance will occasionally send an alert after time zero. For a prediction system with a time horizon of six hours, it is more useful to have alerts occur a mean time of four hours prior to time zero than four minutes prior.

Improving the clinical utility of EWSs requires better measurement of timing. Researchers should incorporate timing into system development, and operational leaders should be cognizant of timing during implementation. Specific steps should include devising better strategies to estimate the relationship of state recognition to clinical time zero and developing methods to discount recognition when it occurs too late to be actionable.

Disclosures

Dr. Rolnick is a consultant to Tuple Health, Inc. and was previously a part-time employee of Acumen, LLC. Dr. Weissman has nothing to disclose.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

A 58-year-old man with a remote history of endocarditis and no prior injection drug use was admitted to the inpatient medicine service with fever and concern for recurrent endocarditis. A transthoracic echocardiogram was unremarkable and the patient remained clinically stable. A transesophageal echocardiogram (TEE) was scheduled for the following morning, but during nursing rounds, the patient was missing from his room. Multiple staff members searched for the patient and eventually located him in the hospital lobby drinking a cup of coffee purchased from the cafeteria. Despite his opposition, he was escorted back to his room and advised to not leave the floor again. Later that day, the patient became frustrated and left the hospital before his scheduled TEE. He was subsequently lost to follow-up.

Patients are admitted to the hospital based upon a medical determination that the patient requires acute observation, evaluation, or treatment. Once admitted, healthcare providers may impose restrictions on the patient’s movement in the hospital, such as restrictions on leaving their assigned floor. Managing the movement of hospitalized patients poses significant challenges for the clinical staff because of the difficulty of providing a treatment environment that ensures safe and efficient delivery of care while promoting patients’ preferences for an unrestrictive environment that respects their independence.^1,2 Broad limits may make it easier for staff to care for patients and reduce concerns about liability, but they may also frustrate patients who may be medically, psychiatrically, and physically stable and do not require stringent monitoring (eg, completing a course of intravenous antibiotics or awaiting placement at outside facilities).

Although this issue has broad implications for patient safety and hospital liability, authoritative guidance and evidence-based literature are lacking. Without clear guidelines, healthcare staff members are likely to spend more time in managing each individual request to leave the floor because they do not have a systematic strategy for making fair and consistent decisions. Here, we describe the patient and institutional considerations when managing patient movement in the hospital. We refer to “patient movement” specifically as a patient’s choice to move to different locations within the hospital, but outside of their assigned room and/or floor. This does not include scheduled, supervised ambulation activities, such as physical therapy.

Practices that promote patient movement offer significant benefits and risks. Enhancing movement is likely to reduce the “physiologic disruption”³ of hospitalization while improving patients’ overall satisfaction and alignment with patient-centered care. Liberalized movement also promotes independence and ambulation that reduces the rate of physical deconditioning.⁴

Despite theoretical benefits, hospitals may be more concerned about adverse events related to patient movement, such as falls, the use of illicit substances, or elopement. Given that hospitals may be legally⁵ and financially responsible⁶ for adverse events associated with patient movement, allowances for off-floor movement should be carefully considered with input from risk management, physicians, nursing leadership, patient advocates, and hospital administration.

Additionally, unannounced movement off the floor may interfere with timely and efficient care by causing lapses in monitoring, such as cardiac telemetry,⁷ medication administration, and scheduled diagnostic tests. In these situations, the risks of patient absence from the floor are significant and may ultimately negate the benefits of continued hospitalization by compromising the central elements of patient care.

Patients’ requests to leave the hospital floor should be evaluated systemically and transparently to promote fair, high-value care. First, a request for liberalized movement should prompt physicians that the patient may no longer require hospitalization and may be ready for the transition to outpatient care.⁸ If the patient still requires inpatient care, then the medical practitioner should make a clinical determination if the patient is medically stable enough to leave their hospital floor. The provider should first identify when the liberalization of movement would be universally inappropriate, such as in patients who are physically unable to ambulate without posing significant harm to themselves. This includes an accidental fall (usually while walking⁵), which is one of the most commonly reported adverse events in an inpatient setting.⁹ Additionally, patients with significant cognitive impairments or those lacking in decision-making capacity may be restricted from leaving their floors unescorted, as they are at a higher risk of disorientation, falls, and death.¹⁰

In determining movement restrictions for patients in isolation, hospitals should refer to the existing guidelines on isolation precautions for the transmission of communicable infections^11,12 and neutropenic precautions.¹³ Additionally, movement restriction for patients who are isolated after screening positive for certain drug-resistant organisms (eg, methicillin-resistant Staphylococcus aureus and vancomycin-resistant enterococci) is controversial and should be evaluated based on the available medical evidence and standards.^14-16

When making a risk-benefit determination about movement, providers should also assess the intent and the potentially unmet needs behind the patient’s request. Patient-centered reasons for enhanced freedom of movement within the hospital include a desire for exercise, greater food choice, and visiting with loved ones, all of which can enable patients to manage the well-known inconveniences and stresses of hospitalization. In contrast, there may be concerns for other intentions behind leaving assigned floors based on the patient’s clinical history, such as the surreptitious use of illicit substances or attempts to elope from the hospital. Advising restriction of movement is justifiable if there is a significant concern for behavior that undermines the safe delivery of care. In patients with active substance use disorders, the appropriate treatment of pain or withdrawal symptoms may better address the patients’ unmet needs, but a lower threshold to restrict movement may be reasonable given the significant risks involved. However, given the widespread stigmatization of patients with substance use disorders,¹⁷ institutional policy and clinicians should adhere to systematic, transparent, and consistent risk assessments for all patients in order to minimize the potential for introducing or exacerbating disparities in care.

In order to work productively with admitted patients, strong practices honor patients’ autonomy by specifying when and how patients are informed of the institution’s expectations about and limitations to inpatient movement. For example, emergency room patients were less likely to elope when treatment expectations were established at the time of presentation by giving them information about wait times and triage protocol.¹⁸ Similarly, by preemptively discussing reasonable restrictions on movement as a part of informed consent for inpatient admission, physicians can establish patients’ expectations early in the admission process and foster a therapeutic alliance on the basis of the shared goals of safe and timely care.

Patients may request or even demand to leave the floor after a healthcare provider has determined that doing so would be unsafe and/or undermine the timely and efficient delivery of care. In these cases, shared decision-making (SDM) can help identify acceptable solutions within the identified constraints. SDM combines the physicians’ experience, expertise, and knowledge of medical evidence with patients’ values, needs, and preferences for care.¹⁹ If patients continue to request to leave the floor after the restriction has been communicated, physicians should discuss whether the current treatment plan should be renegotiated to include a relatively minor modification (eg, a change in the timing or route of administration of medication). If inpatient care cannot be provided safely within the patient’s preferences for movement and attempts to accommodate the patient’s preferences are unsuccessful, then a shift to discharge planning may be appropriate. A summary of this decision process is outlined in the Figure.

In mitigating the burdens of restriction on movement, hospitals may implement a range of options that address patients’ preferences while maintaining safety. Given the potential consequences of liberalized patient movement, it may be prudent to implement these safeguards as a compromise that addresses both the patients’ needs and the hospital’s concerns. These could include an escort for off-floor supervision, timed passes to leave the floor, or volunteers purchasing food for patients from the cafeteria. Creating open, supervised spaces within the hospital (eg, lounges) may also help provide the respite patients need, but in a safe and medically structured environment.

Returning to the introductory case example, we now present an alternative outcome in the context of the practices described above. On the morning of the scheduled TEE, a nurse noted that the patient was missing from his room. Before the staff began searching for the patient, they consulted the medical record which included the admission discussion and agreement to expectations for inpatient movement. The record also included an informed consent discussion indicating the minimal risks of leaving the floor, as the patient could ambulate independently and had no need for continuous monitoring. Finally, a physician’s order authorized the patient to be off the floor until 10 am. The patient returned to his room at 9:45 am and underwent a normal TEE, after which he was discharged home with outpatient follow-up.

The above scenario highlights the benefits of a comprehensive framework for patient movement practices that are transparent, fair, and systematic. Explicitly recognizing competing institutional and patient perspectives can prevent conflict and promote high-quality, safe, efficient, patient-centered care that only restricts the patient’s movement under specified and justifiable conditions. In developing strong hospital practices, institutions should refer to the relevant clinical and ethical standards and draw upon their institutional resources in risk management, clinical staff, and patient advocates.

Acknowledgments

The authors thank Dr. Neil Shapiro and Dr. David Chuquin for their constructive reviews of prior versions of this manuscript.

Disclosures

The authors have no financial conflicts of interest to disclose.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs, the US Government, or the VA National Center for Ethics in Health Care.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Sepsis and heart failure are two common, costly, and deadly conditions. Among hospitalized Medicare patients, these conditions rank as the first and second most frequent principal diagnoses accounting for over $33 billion in spending across all payers.¹ One-third to one-half of all hospital deaths are estimated to occur in patients with sepsis,² and heart failure is listed as a contributing factor in over 10% of deaths in the United States.³

Previous research shows that evidence-based care decisions can impact the outcomes for these patients. For example, sepsis patients receiving intravenous fluids, blood cultures, broad-spectrum antibiotics, and lactate measurement within three hours of presentation have lower mortality rates.⁴ In heart failure, key interventions such as the appropriate use of ACE inhibitors, beta blockers, and referral to disease management programs reduce morbidity and mortality.⁵

However, rapid dissemination and adoption of evidence-based guidelines remain a challenge.^6,7 Policy makers have introduced incentives and penalties to support adoption, with varying levels of success. After four years of Centers for Medicare and Medicaid Services (CMS) penalties for hospitals with excess heart failure readmissions, only 21% performed well enough to avoid a penalty in 2017.⁸ CMS has been tracking sepsis bundle adherence as a core measure, but the rate in 2018 sat at just 54%.⁹ It is clear that new solutions are needed.¹⁰

AdventHealth (formerly Adventist Health System) is a growing, faith-based health system with hospitals across nine states. AdventHealth is a national leader in quality, safety, and patient satisfaction but is not immune to the challenges of delivering consistent, evidence-based care across an extensive network. To accelerate system-wide practice change, AdventHealth’s Office of Clinical Excellence (OCE) partnered with QURE Healthcare and Premier, Inc., to implement a physician engagement and care standardization collaboration involving nearly 100 hospitalists at eight facilities across five states.

This paper describes the results of the Adventist QURE Quality Project (AQQP), which used QURE’s validated, simulation-based measurement and feedback approach to engage hospitalists and standardize evidence-based practices for patients with sepsis and heart failure. We documented specific areas of variation identified in the simulations, how those practices changed through serial feedback, and the impact of those changes on real-world outcomes and costs.

Setting

AdventHealth has its headquarters in Altamonte Springs, Florida. It has facilities in nine states, which includes 48 hospitals. The OCE is comprised of physician leaders, project managers, and data analysts who sponsored the project from July 2016 through July 2018.

Study Participants

AdventHealth hospitals were invited to enroll their hospitalists in AQQP; eight AdventHealth hospitals across five states, representing 91 physicians and 16 nurse practitioners/physician’s assistants (APPs), agreed to participate. Participants included both AdventHealth-employed providers and contracted hospitalist groups. Provider participation was voluntary and not tied to financial incentives; however, participants received Continuing Medical Education credit and, if applicable, Maintenance of Certification points through the American Board of Internal Medicine.

Quasi-experimental Design

We used AdventHealth hospitals not participating in AQQP as a quasi-experimental control group. We leveraged this to measure the impact of concurrent secular effects, such as order sets and other system-wide training, that could also improve practice and outcomes in our study.

Study Objectives and Approach

The explicit goals of AQQP were to (1) measure how sepsis and heart failure patients are cared for across AdventHealth using Clinical Performance and Value (CPV) case simulations, (2) provide a forum for hospitalists to discuss clinical variation, and (3) reduce unneeded variation to improve quality and reduce cost. QURE developed 12 CPV simulated patient cases (six sepsis and six heart failure cases) with case-specific evidenced-based scoring criteria tied to national and Advent­Health evidence-based guidelines. AdventHealth order sets were embedded in the cases and accessible by participants as they cared for their patients.

CPV vignettes are simulated patient cases administered online, and have been validated as an accurate and responsive measure of clinical decision-making in both ambulatory^11-13 and inpatient settings.^14,15 Cases take 20-30 minutes each to complete and simulate a typical clinical encounter: taking the medical history, performing a physical examination, ordering tests, making the diagnosis, implementing initial treatment, and outlining a follow-up plan. Each case has predefined, evidence-based scoring criteria for each care domain. Cases and scoring criteria were reviewed by AdventHealth hospitalist program leaders and physician leaders in OCE. Provider responses were double-scored by trained physician abstractors. Scores range from 0%-100%, with higher scores reflecting greater alignment with best practice recommendations.

In each round of the project, AQQP participants completed two CPV cases, received personalized online feedback reports on their care decisions, and met (at the various sites and via web conference) for a facilitated group discussion on areas of high group variation. The personal feedback reports included the participant’s case score compared to the group average, a list of high-priority personalized improvement opportunities, a summary of the cost of unneeded care items, and links to relevant references. The group discussions focused on six items of high variation. Six total rounds of CPV measurement and feedback were held, one every four months.

At the study’s conclusion, we administered a brief satisfaction survey, asking providers to rate various aspects of the project on a five-point Likert scale.

Data

The study used two primary data sources: (1) care decisions made in the CPV simulated cases and (2) patient-level utilization data from Premier Inc.’s QualityAdvisor^TM (QA) data system. QA integrates quality, safety, and financial data from AdventHealth’s electronic medical record, claims data, charge master, and other resources. QualityAdvisor also calculates expected performance for critical measures, including cost per case and length of stay (LOS), based on a proprietary algorithm, which uses DRG classification, severity-of-illness, risk-of-mortality, and other patient risk factors. We pulled patient-level observed and expected data from AQQP qualifying physicians, defined as physicians participating in a majority of CPV measurement rounds. Of the 107 total hospitalists who participated, six providers did not participate in enough CPV rounds, and 22 providers left AdventHealth and could not be included in a patient-level impact analysis. These providers were replaced with 21 new hospitalists who were enrolled in the study and included in the CPV analysis but who did not have patient-level data before AQQP enrollment. Overall, 58 providers met the qualifying criteria to be included in the impact analysis. We compared their performance to a group of 96 hospitalists at facilities that were not participating in the project. Comparator facilities were selected based on quantitative measures of size and demographic matching the AQQP-facilities ensuring that both sets of hospitals (comparator and AQQP) exhibited similar levels of engagement with Advent- Health quality activities such as quality dashboard performance and order set usage. Baseline patient-level cost and LOS data covered from October 2015 to June 2016 and were re-measured annually throughout the project, from July 2016 to June 2018.

Statistical Analyses

We analyzed three primary outcomes: (1) general CPV-measured improvements in each round (scored against evidence-based scoring criteria); (2) disease-specific CPV improvements over each round; and (3) changes in patient-level outcomes and economic savings among AdventHealth pneumonia/sepsis and heart failure patients from the aforementioned improvements. We used Student’s t-test to analyze continuous outcome variables (including CPV, cost of care, and length of stay data) and Fisher’s exact test for binary outcome data. All statistical analyses were performed using Stata 14.2 (StataCorp LLC, College Station, Texas).

Baseline Characteristics and Assessment

A total of 107 AdventHealth hospitalists participated in this study (Appendix Table 1). 78.1% of these providers rated the organization’s focus on quality and lowering unnecessary costs as either “good” or “excellent,” but 78.8% also reported that variation in care provided by the group was “moderate” to “very high”.

At baseline, we observed high variability in the care of pneumonia patients with sepsis (pneumonia/sepsis) and heart failure patients as measured by the care decisions obtained in the CPV cases. The overall quality score, which is a weighted average across all domains, averaged 61.9% ± 10.5% for the group (Table 1). Disaggregating scores by condition, we found an average overall score of 59.4% ± 10.9% for pneumonia/sepsis and 64.4% ± 9.4% for heart failure. The diagnosis and treatment domains, which require the most clinical judgment, had the lowest average domain scores of 53.4% ± 20.9% and 51.6% ± 15.1%, respectively.

Changes in CPV Scores

To determine the impact of serial measurement and feedback, we compared performance in the first two rounds of the project with the last two rounds. We found that overall CPV quality scores showed a 4.8%-point absolute improvement (P < .001; Table 1). We saw improvements in all care domains, and those increases were significant in all but the workup (P = .470); the most significant increase was in diagnostic accuracy (+19.1%; P < .001).

By condition, scores showed similar, statistically significant overall improvements: +4.4%-points for pneumonia/sepsis (P = .001) and +5.5%-points for heart failure (P < .001) driven by increases in the diagnosis and treatment domains. For example, providers increased appropriate identification of HF severity by 21.5%-points (P < .001) and primary diagnosis of pneumonia/sepsis by 3.6%-points (P = .385).

In the treatment domain, which included clinical decisions related to initial management and follow-up care, there were several specific improvements. For HF, we found that performing all the essential treatment elements—prescribing diuretics, ACE inhibitors and beta blockers for appropriate patients—improved by 13.9%-points (P = .038); ordering VTE prophylaxis increased more than threefold, from 16.6% to 51.0% (P < .001; Table 2). For pneumonia/sepsis patients, absolute adherence to all four elements of the 3-hour sepsis bundle improved by 11.7%-points (P = .034). We also saw a decrease in low-value diagnostic workup items for patient cases in which the guidelines suggest they are not needed, such as urinary antigen testing, which declined by 14.6%-points (P = .001) and sputum cultures, which declined 26.4%-points (P = .004). In addition, outlining an evidence-based discharge plan including a follow-up visit, patient education and medication reconciliation improved, especially for pneumonia/sepsis patients by 24.3%-points (P < .001).

Patient-Level Impact on LOS and Cost Per Case

We took advantage of the quasi-experimental design, in which only a portion of AdventHealth facilities participated in the project, to compare patient-level results from AQQP-participating physicians against the engagement-matched cohort of hospitalists at nonparticipating AdventHealth facilities. We adjusted for potential differences in patient-level case mix between the two groups by comparing the observed/expected (O/E) LOS and cost per case ratios for pneumonia/sepsis and heart failure patients.

At baseline, AQQP-hospitalists performed better on geometric LOS versus the comparator group (O/E of 1.13 vs 1.22; P = .006) but at about the same on cost per case (O/E of 1.16 vs 1.14; P = .390). Throughout the project, as patient volumes and expected per patient costs rose for both groups, O/E ratios improved among both AQQP and non-AQQP providers.

To set apart the contribution of system-wide improvements from the AQQP project-specific impacts, we applied the O/E improvement rates seen in the comparator group to the AQQP group baseline performance. We then compared that to the actual changes seen in the AQQP throughout the project to see if there was any additional benefit from the simulation measurement and feedback (Figure).

Provider Satisfaction

At the project conclusion, we administered a brief survey. Participants were asked to rate aspects of the project (a five-point Likert scale with five being the highest), and 24 responded. The mean ratings of the relevance of the project to their practice and the overall quality of the material were 4.5 and 4.2, respectively. Providers found the individual feedback reports (3.9) slightly more helpful than the webcast group discussions (3.7; Appendix Table 2 ).

As health systems expand, the opportunity to standardize clinical practice within a system has the potential to enhance patient care and lower costs. However, achieving these goals is challenging when providers are dispersed across geographically separated sites and clinical decision-making is difficult to measure in a standardized way.^16,17 We brought together over 100 physicians and APPs from eight different-sized hospitals in five different states to prospectively determine if we could improve care using a standardized measurement and feedback system. At baseline, we found that care varied dramatically among providers. Care varied in terms of diagnostic accuracy and treatment, which directly relate to care quality and outcomes.⁴ After serial measurement and feedback, we saw reductions in unnecessary testing, more guideline-based treatment decisions, and better discharge planning in the clinical vignettes.

We confirmed that changes in CPV-measured practice translated into lower costs and shorter LOS at the patient level. We further validated the improvements through a quasi-experimental design that compared these changes to those at nonparticipating AdventHealth facilities. We saw more significant cost reductions and decreases in LOS in the simulation-based measurement and feedback cohort with the biggest impact early on. The overall savings to the system, attributable specifically to the AQQP approach, is estimated to be $2.4 million.

One advantage of the online case simulation approach is the ability to bring geographically remote sites together in a shared quality-of-care discussion. The interventions specifically sought to remove barriers between facilities. For example, individual feedback reports allowed providers to see how they compare with providers at other AdventHealth facilities and webcast results discussions enable providers across facilities to discuss specific care decisions.

There were several limitations to the study. While the quasi-experimental design allowed us to make informative comparisons between AQQP-participating facilities and nonparticipating facilities, the assignments were not random, and participants were generally from higher performing hospital medicine groups. The determination of secular versus CPV-related improvement is confounded by other system improvement initiatives that may have impacted cost and LOS results. This is mitigated by the observation that facilities that opted to participate performed better at baseline in risk-adjusted LOS but slightly worse in cost per case, indicating that baseline differences were not dramatic. While both groups improved over time, the QURE measurement and feedback approach led to larger and more rapid gains than those seen in the comparator group. However, we could not exclude the potential that project participation at the site level was biased to those groups disposed to performance improvement. In addition, our patient-level data analysis was limited to the metrics available and did not allow us to directly compare patient-level performance across the plethora of clinically relevant CPV data that showed improvement. Our inpatient cost per case analysis showed significant savings for the system but did not include all potentially favorable economic impacts such as lower follow-up care costs for patients, more accurate reimbursement through better coding or fewer lost days of productivity.

With continued consolidation in healthcare and broader health systems spanning multiple geographies, new tools are needed to support standardized, evidence-based care across sites. This standardization is especially important, both clinically and financially, for high-volume, high-cost diseases such as sepsis and heart failure. However, changing practice cannot happen without collaborative engagement with providers. Standardized patient vignettes are an opportunity to measure and provide feedback in a systematic way that engages providers and is particularly well-suited to large systems and common clinical conditions. This analysis, from a real-world study, shows that an approach that standardizes care and lowers costs may be particularly helpful for large systems needing to bring disparate sites together as they concurrently move toward value-based payment.

Disclosures

QURE, LLC, whose intellectual property was used to prepare the cases and collect the data, was contracted by AdventHealth. Otherwise, any of the study authors report no potential conflicts to disclose.

Funding

This work was funded by a contract between AdventHealth (formerly Adventist Health System) and QURE, LLC.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

Approximately 60,000 hospitalists were working in the United States in 2018.¹ Hospitalist groups work collaboratively because of the shiftwork required for 24/7 patient coverage, and first-rate clinical documentation is essential for quality care.² Thoughtful clinical documentation not only transmits one provider’s clinical reasoning to other providers but is a professional responsibility.³ Hospitalists spend two-thirds of their time in indirect patient-care activities and approximately one quarter of their time on documentation in electronic health records (EHRs).⁴ Despite documentation occupying a substantial portion of the clinician’s time, published literature on the best practices for the documentation of clinical reasoning in hospital medicine or its assessment remains scant.^5-7

Clinical reasoning involves establishing a diagnosis and developing a therapeutic plan that fits the unique circumstances and needs of the patient.⁸ Inpatient providers who admit patients to the hospital end the admission note with their assessment and plan (A&P) after reflecting about a patient’s presenting illness. The A&P generally represents the interpretations, deductions, and clinical reasoning of the inpatient providers; this is the section of the note that fellow physicians concentrate on over others.⁹ The documentation of clinical reasoning in the A&P allows for many to consider how the recorded interpretations relate to their own elucidations resulting in distributed cognition.¹⁰

Disorganized documentation can contribute to cognitive overload and impede thoughtful consideration about the clinical presentation.³ The assessment of clinical documentation may translate into reduced medical errors and improved note quality.^11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students.^13-15 The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists “do”; if this evolves into a mechanism for offering formative feedback, such professional development would impact the highest level of Miller’s assessment pyramid.¹⁶ We therefore undertook this study to establish a metric to assess the hospitalist providers’ documentation of clinical reasoning in the A&P of an admission note.

Study Design, Setting, and Subjects

This was a retrospective study that reviewed the admission notes of hospitalists for patients admitted over the period of January 2014 and October 2017 at three hospitals in Maryland. One is a community hospital (Hospital A) and two are academic medical centers (Hospital B and Hospital C). Even though these three hospitals are part of one health system, they have distinct cultures and leadership, serve different populations, and are staffed by different provider teams.

The notes of physicians working for the hospitalist groups at each of the three hospitals were the focus of the analysis in this study.

Development of the Documentation Assessment Rubric

A team was assembled to develop the Clinical Reasoning in Admission Note Assessment & PLan (CRANAPL) tool. The CRANAPL was designed to assess the comprehensiveness and thoughtfulness of the clinical reasoning documented in the A&P sections of the notes of patients who were admitted to the hospital with an acute illness. Validity evidence for CRANAPL was summarized on the basis of Messick’s unified validity framework by using four of the five sources of validity: content, response process, internal structure, and relations to other variables.¹⁷

Content Validity

The development team consisted of members who have an average of 10 years of clinical experience in hospital medicine; have studied clinical excellence and clinical reasoning; and have expertise in feedback, assessment, and professional development.^18-22 The development of the CRANAPL tool by the team was informed by a review of the clinical reasoning literature, with particular attention paid to the standards and competencies outlined by the Liaison Committee on Medical Education, the Association of American Medical Colleges, the Accreditation Council on Graduate Medical Education, the Internal Medicine Milestone Project, and the Society of Hospital Medicine.^23-26 For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool’s development were Baker’s Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool;¹⁴ King’s Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric;¹⁵ and three other studies related to diagnostic reasoning.^16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development. The CRANAPL tool was shown to colleagues at other institutions who are leaders on clinical reasoning and was presented at academic conferences in the Division of General Internal Medicine and the Division of Hospital Medicine of our institution. Feedback resulted in iterative revisions. The aforementioned methods established content validity evidence for the CRANAPL tool.

Response Process Validity

Several of the authors pilot-tested earlier iterations on admission notes that were excluded from the sample when refining the CRANAPL tool. The weaknesses and sources of confusion with specific items were addressed by scoring 10 A&Ps individually and then comparing data captured on the tool. This cycle was repeated three times for the iterative enhancement and finalization of the CRANAPL tool. On several occasions when two authors were piloting the near-final CRANAPL tool, a third author interviewed each of the two authors about reactivity while assessing individual items and exploring with probes how their own clinical documentation practices were being considered when scoring the notes. The reasonable and thoughtful answers provided by the two authors as they explained and justified the scores they were selecting during the pilot testing served to confer response process validity evidence.

Finalizing the CRANAPL Tool

The nine-item CRANAPL tool includes elements for problem representation, leading diagnosis, uncertainty, differential diagnosis, plans for diagnosis and treatment, estimated length of stay (LOS), potential for upgrade in status to a higher level of care, and consideration of disposition. Although the final three items are not core clinical reasoning domains in the medical education literature, they represent clinical judgments that are especially relevant for the delivery of the high-quality and cost-effective care of hospitalized patients. Given that the probabilities and estimations of these three elements evolve over the course of any hospitalization on the basis of test results and response to therapy, the documentation of initial expectations on these fronts can facilitate distributed cognition with all individuals becoming wiser from shared insights.¹⁰ The tool uses two- and three-point rating scales, with each number score being clearly defined by specific written criteria (total score range: 0-14; Appendix).

Data Collection

Hospitalists’ admission notes from the three hospitals were used to validate the CRANAPL tool. Admission notes from patients hospitalized to the general medical floors with an admission diagnosis of either fever, syncope/dizziness, or abdominal pain were used. These diagnoses were purposefully examined because they (1) have a wide differential diagnosis, (2) are common presenting symptoms, and (3) are prone to diagnostic errors.^29-32

The centralized EHR system across the three hospitals identified admission notes with one of these primary diagnoses of patients admitted over the period of January 2014 to October 2017. We submitted a request for 650 admission notes to be randomly selected from the centralized institutional records system. The notes were stratified by hospital and diagnosis. The sample size of our study was comparable with that of prior psychometric validation studies.^33,34 Upon reviewing the A&Ps associated with these admissions, 365 notes were excluded for one of three reasons: (1) the note was written by a nurse practitioner, physician assistant, resident, or medical student; (2) the admission diagnosis had been definitively confirmed in the emergency department (eg, abdominal pain due to diverticulitis seen on CT); and (3) the note represented the fourth or more note by any single provider (to sample notes of many providers, no more than three notes written by any single provider were analyzed). A total of 285 admission notes were ultimately included in the sample.

Data were deidentified, and the A&P sections of the admission notes were each copied from the EHR into a unique Word document. Patient and hospital demographic data (including age, gender, race, number of comorbid conditions, LOS, hospital charges, and readmission to the same health system within 30 days) were collected separately from the EHR. Select physician characteristics were also collected from the hospitalist groups at each of the three hospitals, as was the length (word count) of each A&P.

The study was approved by our institutional review board.

Data Analysis

Two authors scored all deidentified A&Ps by using the finalized version of the CRANAPL tool. Prior to using the CRANAPL tool on each of the notes, these raters read each A&P and scored them by using two single-item rating scales: a global clinical reasoning and a global readability/clarity measure. Both of these global scales used three-item Likert scales (below average, average, and above average). These global rating scales collected the reviewers’ gestalt about the quality and clarity of the A&P. The use of gestalt ratings as comparators is supported by other research.³⁵

Descriptive statistics were computed for all variables. Each rater rescored a sample of 48 records (one month after the initial scoring) and intraclass correlations (ICCs) were computed for intrarater reliability. ICCs were calculated for each item and for the CRANAPL total to determine interrater reliability.

The averaged ratings from the two raters were used for all other analyses. For CRANAPL’s internal structure validity evidence, Cronbach’s alpha was calculated as a measure of internal consistency. For relations to other variables validity evidence, CRANAPL total scores were compared with the two global assessment variables with linear regressions.

Bivariate analyses were performed by applying parametric and nonparametric tests as appropriate. A series of multivariate linear regressions, controlling for diagnosis and clustered variance by hospital site, were performed using CRANAPL total as the dependent variable and patient variables as predictors.

All data were analyzed using Stata (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, Texas: StataCorp LP.)

The admission notes of 120 hospitalists were evaluated (Table 1). A total of 39 (33%) physicians were moonlighters with primary appointments outside of the hospitalist division, and 81 (68%) were full-time hospitalists. Among the 120 hospitalists, 48 (40%) were female, 60 (50%) were international medical graduates, and 90 (75%) were of nonwhite race. Most hospitalist physicians (n = 47, 58%) had worked in our health system for less than five years, and 64 hospitalists (53%) devoted greater than 50% of their time to patient care.

Approximately equal numbers of patient admission notes were pulled from each of the three hospitals. The average age of patients was 67.2 (SD 13.6) years, 145 (51%) were female, and 120 (42%) were of nonwhite race. The mean LOS for all patients was 4.0 (SD 3.4) days. A total of 44 (15%) patients were readmitted to the same health system within 30 days of discharge. None of the patients died during the incident hospitalization. The average charge for each of the hospitalizations was $10,646 (SD $9,964).

CRANAPL Data

Figure 1 shows the distribution of the scores given by each rater for each of the nine items. The mean of the total CRANAPL score given by both raters was 6.4 (SD 2.2). Scoring for some items were high (eg, summary statement: 1.5/2), whereas performance on others were low (eg, estimating LOS: 0.1/1 and describing the potential need for upgrade in care: 0.0/1).

Validity of the CRANAPL Tool’s Internal Structure

Cronbach’s alpha, which was used to measure internal consistency within the CRANAPL tool, was 0.43. The ICC, which was applied to measure the interrater reliability for both raters for the total CRANAPL score, was 0.83 (95% CI:  0.76-0.87). The ICC values for intrarater reliability for raters 1 and 2 were 0.73 (95% CI: 0.60-0.83) and 0.73 (95% CI: 0.45-0.86), respectively.

Relations to Other Variables Validity

Associations between CRANAPL total scores, global clinical reasoning, and global scores for note readability/clarity were statistically significant (P < .001), Figure 2.

Eight out of nine CRANAPL variables were statistically significantly different across the three hospitals (P <. 01) when data were analyzed by hospital site. Hospital C had the highest mean score of 7.4 (SD 2.0), followed by Hospital B with a score of 6.6 (SD 2.1), and Hospital A had the lowest total CRANAPL score of 5.2 (SD 1.9). This difference was statistically significant (P < .001). Five variables with respect to admission diagnoses (uncertainty acknowledged, differential diagnosis, plan for diagnosis, plan for treatment, and upgrade plan) were statistically significantly different across notes. Notes for syncope/dizziness generally yielded higher scores than those for abdominal pain and fever.

Factors Associated with High CRANAPL Scores

Table 2 shows the associations between CRANAPL scores and several covariates. Before adjustment, high CRANAPL scores were associated with high word counts of A&Ps (P < .001) and high hospital charges (P < .05). These associations were no longer significant after adjusting for hospital site and admitting diagnoses.

We reviewed the documentation of clinical reasoning in 285 admission notes at three different hospitals written by hospitalist physicians during routine clinical care. To our knowledge, this is the first study that assessed the documentation of hospitalists’ clinical reasoning with real patient notes. Wide variability exists in the documentation of clinical reasoning within the A&Ps of hospitalists’ admission notes. We have provided validity evidence to support the use of the user-friendly CRANAPL tool.

Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students.^14,15 The ICCs for the IDEA rubric used to assess medical students’ documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students’ documentation of pediatric history and physical notes.¹⁵ These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making.^36-39 CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual’s assessment over time. The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning.^13,,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy;³⁵ and effective documentation promotes communication and the coordination of care among the members of the care team.³

The total CRANAPL scores varied by hospital site with academic hospitals (B and C) scoring higher than the community hospital (A) in our study. Similarly, lengthy A&Ps were associated with high CRANAPL scores (P < .001) prior to adjustment for hospital site. Healthcare providers consider that the thoroughness of documentation denotes quality and attention to detail.^35,42 Comprehensive documentation takes time; the longer notes by academic hospitalists than those by community hospitalists may be attributed to the fewer number of patients generally carried by hospitalists at academic centers than that by hospitalists at community hospitals.⁴³

The documentation of the estimations of LOS, possibility of potential upgrade, and thoughts about disposition were consistently poorly described across all hospital sites and diagnoses. In contrast to CRANAPL, other clinical reasoning rubrics have excluded these items or discussed uncertainty.^14,15,44 These elements represent the forward thinking that may be essential for high-quality progressive care by hospitalists. Physicians’s difficulty in acknowledging uncertainty has been associated with resource overuse, including the excessive ordering of tests, iatrogenic injury, and heavy financial burden on the healthcare system.^45,46 The lack of thoughtful clinical and management reasoning at the time of admission is believed to be associated with medical errors.⁴⁷ If used as a guide, the CRANAPL tool may promote reflection on the part of the admitting physician. The estimations of LOS, potential for upgrade to a higher level of care, and disposition are markers of optimal inpatient care, especially for hospitalists who work in shifts with embedded handoffs. When shared with colleagues (through documentation), there is the potential for distributed cognition¹⁰ to extend throughout the social network of the hospitalist group. The fact that so few providers are currently including these items in their A&P’s show that the providers are either not performing or documenting the ‘reasoning’. Either way, this is an opportunity that has been highlighted by the CRANAPL tool.

Several limitations of this study should be considered. First, the CRANAPL tool may not have captured elements of optimal clinical reasoning documentation. The reliance on multiple methods and an iterative process in the refinement of the CRANAPL tool should have minimized this. Second, this study was conducted across a single healthcare system that uses the same EHR; this EHR or institutional culture may influence documentation practices and behaviors. Given that using the CRANAPL tool to score an A&P is quick and easy, the benefit of giving providers feedback on their notes remains to be seen—here and at other hospitals. Third, our sample size could limit the generalizability of the results and the significance of the associations. However, the sample assessed in our study was significantly larger than that assessed in other studies that have validated clinical reasoning rubrics.^14,15 Fourth, clinical reasoning is a broad and multidimensional construct. The CRANAPL tool focuses exclusively on hospitalists’ documentation of clinical reasoning and therefore does not assess aspects of clinical reasoning occurring in the physicians’ minds. Finally, given our goal to optimally validate the CRANAPL tool, we chose to test the tool on specific presentations that are known to be associated with diagnostic practice variation and errors. We may have observed different results had we chosen a different set of diagnoses from each hospital. Further validity evidence will be established when applying the CRANPL tool to different diagnoses and to notes from other clinical settings.

In conclusion, this study focuses on the development and validation of the CRANAPL tool that assesses how hospitalists document their clinical reasoning in the A&P section of admission notes. Our results show that wide variability exists in the documentation of clinical reasoning by hospitalists within and across hospitals. Given the CRANAPL tool’s ease-of-use and its versatility, hospitalist divisions in academic and nonacademic settings may use the CRANAPL tool to assess and provide feedback on the documentation of hospitalists’ clinical reasoning. Beyond studying whether physicians can be taught to improve their notes with feedback based on the CRANAPL tool, future studies may explore whether enhancing clinical reasoning documentation may be associated with improvements in patient care and clinical outcomes.

Acknowledgments

Dr. Wright is the Anne Gaines and G. Thomas Miller Professor of Medicine which is supported through Hopkins’ Center for Innovative Medicine.

The authors thank Christine Caufield-Noll, MLIS, AHIP (Johns Hopkins Bayview Medical Center, Baltimore, Maryland) for her assistance with this project.

Disclosures

The authors have nothing to disclose.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

A 56-year-old Japanese man with a history of renal transplantation 20 years prior presented to the emergency department (ED) with two months of dyspnea on exertion and one day of fever and chills. The patient was in his usual state of health until two months prior to presentation, when he gradually noticed shortness of breath after sustained or effortful physical activities. The dyspnea improved with rest. Over the following two months, he noticed that the shortness of breath came on with lesser degrees of exertion, such as walking 100 meters. One day before presentation, he developed a fever of 39°C and chills at home, which prompted him to seek ED care. He denied chest pain, cough, leg swelling, or paroxysmal nocturnal dyspnea.

The differential diagnosis of exertional dyspnea progressing over several months includes cardiac, pulmonary, hematologic, and neuromuscular conditions. The patient’s history of renal transplantation prompts consideration of worsening indolent pneumonia (eg, Aspergillus, cytomegalovirus [CMV], or Pneumocystis pneumonia), allograft dysfunction with volume overload, recrudescence of the underlying disease that incited renal failure earlier in life (eg, vasculitis), or a late-onset posttransplantation lymphoproliferative disorder (PTLD). Additionally, acute fever in an immunocompromised patient immediately raises suspicion for infection (eg, pneumonia, enteritis, or urinary tract infection). At this point, it is difficult to know whether the subacute-to-chronic exertional dyspnea and the acute fever are consequences of the same disease or separate, potentially overlapping, processes.

His past medical history was significant for end-stage renal disease due to membranoproliferative glomerular nephropathy (MPGN), for which living, related-donor kidney transplantation was performed 20 years earlier. He also had type 2 diabetes mellitus, hypertension, and basal cell carcinoma of the face, which had been resected three years prior without spread or recurrence. He had no known allergies. Medications included prednisolone 15 mg daily, azathioprine 100 mg daily, and cyclosporine 100 mg daily, as well as amlodipine and candesartan. He lived in Japan with his wife and children. He denied any animal or environmental exposures. He did not smoke cigarettes or drink alcohol and had not traveled recently. His father had diabetes mellitus.

Recrudescence of an underlying autoimmune condition that may have incited MPGN earlier in life is unlikely while taking an immunosuppressive regimen consisting of prednisolone, azathioprine, and cyclosporine. However, these medications do increase susceptibility to infections, lymphoma, and skin cancers. Though he is immunocompromised, the patient is not on prophylaxis for Pneumocystis pneumonia (PCP). PCP in HIV-negative patients is associated with recent glucocorticoid exposure and typically follows an acute-to-subacute course with hypoxemia and respiratory distress. Though the risk of PCP infection is considered highest in the early posttransplantation period (when immunosuppression is most intense), many cases are diagnosed years after transplantation among patients no longer on prophylaxis. The patient has type 2 diabetes mellitus and hypertension, which are known complications of calcineurin inhibitor and steroid therapy and increase the risk of cardiovascular disease. Cardiovascular disease is a major cause of death among renal transplant recipients. Exertional dyspnea may be the presenting symptom of coronary artery disease.

On physical examination, the patient was alert, oriented, and in no acute distress. His temperature was 38.5°C, blood pressure 120/60 mm Hg, heart rate 146 beats per minute, respiratory rate 18 breaths per minute, and oxygen saturation 93% while breathing ambient air. The conjunctiva were normal without pallor or icterus. There was no cervical lymphadenopathy. Cardiac examination revealed tachycardia with a regular rhythm, normal S1 and S2, and no murmurs, rubs, or gallops. Jugular venous pressure was not elevated, and there was no lower extremity edema. Lungs were clear to auscultation bilaterally. The abdomen was soft, nontender, and nondistended. There was no tenderness over the transplanted kidney and no hepatosplenomegaly.

Dyspnea, fever, and tachycardia may be the sole manifestations of pneumonia in solid organ transplant recipients. The absence of cough or adventitious breath sounds does not eliminate concern for pneumonia. Pathogens that cause indolent pneumonia in immunocompromised patients include viruses (such as typical respiratory viruses and CMV), bacteria (typical organisms, Nocardia, Rhodococcus), and mycobacteria. Fungal causes include Aspergillus, Candida, Cryptococcus, Pneumocystis, and endemic mycoses. A detailed environmental history should be taken, and providers should ascertain which fungal diseases are endemic in the patient’s region of residence. There are no examination features suggesting hypervolemia or anemia. Although there is no hepatosplenomegaly or lymphadenopathy, PTLD often involves extranodal tissues, including the lungs. The incidence of PTLD is highest in the 12 months following transplantation, but it may occur at any time in the posttransplantation course. A complete blood count, comprehensive metabolic panel, lactate dehydrogenase (LDH), and blood and sputum cultures are indicated, along with computed tomography (CT) of the chest.

The leukocyte count was 3,500 cells/mm³, the hemoglobin level 9.0 g/dL, mean corpuscular volume 102 fL, and the platelet count 137,000 cells/mm³. The sodium level was 130 mEq/L, potassium 4.6 mEq/L, blood urea nitrogen 41 mg/dL, and creatinine 3.5 mg/dL. These complete blood count and serum electrolyte results were unchanged from the patient’s baseline values. The serum LDH level was 1,895 IU/L (normal range, 115-245 IU/L). The serum ferritin was 2,114 ng/mL (normal range, 13-277 ng/mL). A chest radiograph revealed diffuse, airspace-filling opacities in the bilateral lung bases. The urinalysis was normal. The patient was admitted and started empirically on intravenous ceftriaxone for potential bacterial pneumonia.

Chronic pancytopenia may result from azathioprine or cyclosporine use, marrow suppression or infiltration by a multisystem disease, or nutritional deficiency. Hemophagocytic lymphohistiocytosis (HLH) triggered by infection, a rheumatologic condition, acquired immunodeficiency, or malignancy can present with fevers, pancytopenia, and elevated ferritin, while splenomegaly may be absent. The euvolemic state, baseline creatinine level, and normal urinalysis argue against allograft dysfunction. The elevated serum ferritin nonspecifically confirms systemic inflammation. LDH, an intracellular enzyme involved in the bidirectional conversion of lactate to pyruvate, is expressed across tissue types. Elevated serum LDH attests to cell destruction, in this case potentially from lung infection (such as PCP) or malignancy (such as PTLD). At this point, the differential diagnosis of fever and pulmonary infiltrates in this patient remains broad.

Azathioprine and cyclosporine were stopped. The patient remained febrile despite the administration of intravenous antibiotics. His hypoxia worsened with an oxygen saturation of 90%-93% on 5 L/min of supplemental oxygen administered by nasal cannula. Noncontrast chest CT obtained on the second hospital day revealed ground-glass opacities in the bilateral lung bases. Blood, sputum, and urine cultures were sterile. As empiric therapies, ganciclovir was started for CMV infection, ciprofloxacin added for atypical pneumonia, and trimethoprim-sulfamethoxazole added for Pneumocystis infection.

These chest imaging findings help prioritize the differential diagnosis. Bibasilar ground-glass opacities are evident, while pulmonary masses, nodules, cavitation, adenopathy, and pleural effusions are absent. The differential diagnosis of multifocal ground-glass opacities on chest imaging includes infectious pneumonia, chronic interstitial lung disease, acute alveolar conditions (eg, cardiogenic pulmonary edema, acute respiratory distress syndrome, diffuse alveolar hemorrhage), or other pathologies (eg, drug toxicity, bronchoalveolar carcinoma, cryptogenic organizing pneumonia).

Infectious pneumonia is the principal concern. A diagnosis of PCP could be unifying, given dyspnea, progressive respiratory failure with hypoxia, and elevated LDH in an immunocompromised patient who is not prescribed PCP prophylaxis. The bilateral lung infiltrates and the absence of thoracic adenopathy or pleural effusions are characteristic of PCP as well. However, caution should be exercised in making specific infectious diagnoses in immunocompromised hosts on the basis of clinical and imaging findings alone. There can be overlap in the radiologic appearance of various infections (eg, CMV pneumonia can also present with bilateral ground-glass infiltrates, with concurrent fever, hypoxia, and pancytopenia). Additionally, more than one pneumonic pathogen may be implicated (eg, acute viral pneumonia superimposed on indolent fungal pneumonia). Polymerase chain reaction (PCR) analysis of respiratory secretions for viruses, serum PCR and serologic testing for herpes viruses, and serum beta-D-glucan and galactomannan assays are indicated. Serum serologic testing for fungi and bacteria such as Nocardia can be helpful, though the negative predictive values of these tests may be reduced in patients with impaired humoral immunity. Timely bronchoalveolar lavage (BAL) with microbiologic and PCR analysis and cytology is advised.

Fever, elevated LDH, cytopenias, and pulmonary infiltrates also raise suspicion for an underlying hematologic malignancy, such as PTLD. However, pulmonary PTLD is seen more often in lung transplant recipients than in patients who have undergone transplantation of other solid organs. In kidney transplant recipients, PTLD most commonly manifests in the allograft itself, gastrointestinal tract, central nervous system, or lymph nodes; lung involvement is less common. Chest imaging in affected patients may reveal nodular or reticulonodular infiltrates of basilar predominance, solitary or multiple masses, cavitating or necrotic lesions, and/or lymphadenopathy. In this patient who has undergone renal transplantation, late-onset PTLD with isolated pulmonary involvement, with only ground-glass opacities on lung imaging, would be an atypical presentation of an uncommon syndrome.

Despite empiric treatment with antibiotics and antiviral agents, the patient’s fever persisted. His respiratory rate increased to 30 breaths per minute. His hypoxia worsened, and he required nasal cannula high-flow oxygen supplementation at 30 L/min with a fraction of inspired oxygen (FiO₂) of 40%. On the fifth hospital day, contrast CT scan of the chest and abdomen showed new infiltrates in the bilateral upper lung fields as well as an area of low density in the tail of the pancreas without a focal mass (Figure 1). At this point, BAL was performed, and fluid PCR analysis returned positive for Pneumocystis jirovecii. CMV direct immunoperoxidase staining of leukocytes with peroxidase-labeled monoclonal antibody (C7-HRP test) was positive at five cells per 7.35 × 10⁴ peripheral blood leukocytes. The serum Epstein-Barr virus (EBV) viral capsid antigen (VCA) IgG was positive, while VCA IgM and EBV nuclear antigen IgG were negative. A bone marrow biopsy revealed mild hemophagocytosis. His serum soluble interleukin-2 (sIL2R) level was elevated at 5,254 U/mL (normal range, 122-496 U/mL). Given the BAL Pneumocystis PCR result, the dose of prednisolone was increased to 30 mg/day, and the patient’s fever subsided. Supplemental oxygen was weaned to an FiO₂ of 35%.

A CMV antigenemia assay returned positive, suggesting prior CMV infection. However, to diagnose CMV pneumonia, the virus must be detected in BAL fluid by PCR or cytologic analysis. CMV infection has been associated with cytopenias, HLH, pancreatic infiltration, and an increased risk for fungal infections and EBV-related PTLD. CMV infection could explain the first phase of this patient’s illness. Serum and BAL PCR for CMV are advised. Meanwhile, EBV testing indicates prior infection but does not distinguish between recent or more distant infection. EBV has been implicated in the pathophysiology of PTLD, as EBV-infected lymphoid tissue may proliferate in a variety of organs under reduced T-cell surveillance. EBV infection or PTLD with resulting immunomodulation may pose other explanations for this patient’s development of PCP infection. Cytologic analysis of the BAL fluid and marrow aspirate for evidence of PTLD is warranted. Finally, CMV, EBV, and PTLD have each been reported to trigger HLH. Though this patient has fevers, mild marrow hemophagocytosis, elevated serum ferritin, and elevated serum IL-2 receptor levels, he does not meet other diagnostic criteria for HLH (such as more pronounced cytopenias, splenomegaly, hypertriglyceridemia, hypofibrinogenemia, and low or absent natural killer T-cell activity). However, HLH may be muted in this patient because he was prescribed cyclosporine, which has been used in HLH treatment protocols.

On the 11^th hospital day, the patient developed hemorrhagic shock due to massive hematemesis and was transferred to the intensive care unit. His hemoglobin level was 5.9 g/dL. A total of 18 units of packed red blood cells were transfused over the following week for ongoing gastrointestinal bleeding. The serum LDH level increased to 4,139 IU/L, and the ferritin level rose to 7,855 ng/mL. The EBV copy level by serum PCR returned at 1 × 10⁶ copies/mL (normal range, less than 2 x 10² copies/mL). The patient was started on methylprednisolone (1 g/day for three days) and transitioned to dexamethasone and cyclosporine for possible EBV-related HLH. Ceftazidime, vancomycin, trimethoprim-sulfamethoxazole, and ciprofloxacin were administered. Amphotericin-B was initiated empirically for potential fungal pneumonia. Ganciclovir was continued. However, the patient remained in shock despite vasopressors and transfusions and died on the 22^nd hospital day.

The patient deteriorated despite broad antimicrobial therapy. Laboratory studies revealed EBV viremia and rising serum LDH. Recent EBV infection may have induced PTLD in the gastrointestinal tract, which is a commonly involved site among affected renal transplant patients. Corticosteroids and stress from critical illness can contribute to intestinal mucosal erosion and bleeding from a luminal PTLD lesion. Overall, the patient’s condition was likely explained by EBV infection, which triggered HLH and gastrointestinal PTLD. The resulting immunomodulation increased his risk for PCP infection beyond that conferred by chronic immunosuppression. It is still possible that he had concomitant CMV pneumonia, Aspergillus pneumonia, or even pulmonary PTLD, in addition to the proven PCP diagnosis.

An autopsy was performed. Atypical lymphocytic infiltration and diffuse alveolar damage were shown in the right upper lobe (Figure 2). EBV RNA-positive atypical lymphocytes coexpressing CD20 were demonstrated in multiple organs including the bone marrow, lungs, heart, stomach, adrenal glands, duodenum, ileum, and mesentery (Figure 3). This confirmed the diagnosis of an underlying EBV-positive posttransplant lymphoproliferative disorder. Serum and BAL CMV PCR assays returned negative. Neither CMV nor Aspergillus was identified in autopsy specimens.

A broad differential diagnosis should be considered when acute fever develops in a patient who has undergone solid organ transplantation. Causes may include opportunistic and nonopportunistic infections as well as noninfectious etiologies such as malignancy, organ rejection, inflammatory conditions, and medication toxicity.^1,2 As the discussant noted, more than one infection, or both infection and malignancy, can coexist in immunocompromised patients. For example, while viral pathogens such as EBV, CMV, and respiratory syncytial virus can cause illness due to direct tissue infection, they can also exert indirect effects in transplant recipients: acting as cofactors for and enabling other infections by causing immunosuppression (eg, Aspergillus or PCP developing after CMV infection), triggering graft rejection by upregulating proinflammatory cytokines, and inducing oncogenesis (eg, EBV-related PTLD).^1,3-5

PTLD is a rare but serious complication of solid organ transplantation and immunosuppression. Most cases are driven by EBV infection and subsequent transformation of infected lymphoid tissue in a variety of organs in the context of reduced T-cell surveillance.⁶ The incidence of PTLD varies based on the organ transplanted, ranging from 0.8%-2.5% in those who have undergone renal transplantation to 1.0%-5.5% in liver transplant recipients and 3.0%-10% in lung transplant recipients.³ The incidence has increased over the past decade. This may be due to a greater number of solid organ transplantations being performed, aging of the transplant donor/recipient population, new immunosuppressive regimens, and improved PTLD diagnosis due to superior diagnostic tools and clinician awareness.³ However, the mortality rate among solid organ transplant recipients with PTLD remains high, ranging from 40% to 70%.⁶

Risk factors for PTLD include a greater intensity of T-cell immunosuppression,⁷ history of pretransplant malignancy, recipient EBV seronegativity and donor seropositivity, and younger age at the time of transplantation.^8-10 EBV-related PTLD incidence in solid organ transplant recipients is greatest in the early posttransplantation course (the period of most intense immunosuppression) with over 80% of cases occurring in the first posttransplant year.¹¹

A high index of suspicion for PTLD is warranted in any solid organ transplant recipient who presents with constitutional symptoms, adenopathy, or cytopenias. Clinical suspicion of PTLD can be informed by risk factors, constitutional symptoms, elevated serum LDH, a detectable or rising serum EBV viral load, and radiologic adenopathy or visceral tissue infiltration.¹² The clinical presentation of PTLD is heterogeneous and varies in accordance with the organs affected. Extranodal involvement, such as pulmonary, gastrointestinal, and bone marrow involvement, is more common in PTLD than in other types of lymphoma.¹³ In this patient, the cytopenias, elevated serum LDH level, lung infiltrates, and radiologic pancreatic tail abnormality served as early clues to the presence of underlying PTLD.

The standard approach to diagnosing PTLD is biopsy of a suspicious lesion (adenopathy or an infiltrated visceral organ) with histopathological examination. Pathology may demonstrate distorted tissue architecture, clonal lymphocytes, or EBV-positive lymphocytes.¹⁴ Conventional CT is the most commonly used imaging modality to detect adenopathy or tissue infiltration related to PTLD,³ though 18F-fluorodeoxyglucose position-emission tomography (FDG-PET) is also used. Although FDG-PET has high diagnostic accuracy, with an overall sensitivity of 89% and specificity of 89%, false-negative results have been reported, particularly in cases of early PTLD lesions and diffuse large B-cell lymphoma.¹⁵ The majority of patients with EBV-associated PTLD demonstrate significant elevations in the serum EBV viral load compared with immunosuppressed controls without PTLD.¹⁶ An elevated EBV viral load can support a diagnosis of PTLD, though the absence of EBV viremia does not rule it out.¹⁷ Some transplant centers perform posttransplantation monitoring of the serum EBV viral load to aid in PTLD risk assessment and early diagnosis.

Management of PTLD is patient-specific and may involve reduction of immunosuppressive therapy, rituximab, chemotherapy, surgical excision, and/or radiation.¹³ Reduction of immunosuppression is the cornerstone of treatment.¹⁸ In patients who do not respond to the reduction of immunosuppression, rituximab and immunochemotherapy are second-line treatment options. A prospective, multicenter phase 2 trial (the PTLD-1 trial) demonstrated a complete response rate of 40% among patients with PTLD managed with rituximab.¹⁹

In summary, this case illustrates the importance of maintaining a broad differential diagnosis when acute fever develops in a patient who has undergone solid organ transplantation. The presence of more than one condition should be considered when the clinical presentation cannot be explained by a single diagnosis, as infections and malignancies can coexist in immunocompromised hosts. This case also highlights an unusual clinical presentation of PTLD, which was heralded mainly by its immunomodulatory effects rather than by compatible symptoms or obvious mass lesions.

Carefully reviewing the patient’s medical history and understanding how it sets the stage for the present illness is an essential step in clinical problem solving, because what is past is prologue.

• Fever in solid organ transplant recipients should prompt consideration of a broad differential diagnosis, including infection, malignancy, organ graft rejection, autoimmune disease, and medication toxicity.
• PTLD is a rare but serious complication of organ transplantation. Most cases are driven by EBV infection and transformation of infected lymphocytes in a variety of organs in the context of reduced T-cell surveillance. The clinical presentation can be heterogeneous and varies depending on the organs and tissues involved.
• More than one infection, or both infection and malignancy, can coexist in organ transplant recipients. Viral pathogens can exert direct pathologic effects on tissue but can also exert indirect effects, such as contributing to opportunistic infection susceptibility, graft rejection, and oncogenesis.

Disclosures

The authors have nothing to disclose.

Previous Publication

This case was originally reported in the 121st Okinawa Association of Medical Sciences in 2015 in Okinawa, Japan, and the conference abstracts were covered in The Okinawa Medical Journal. The publication did not provide any detailed, step-by-step analysis of clinical decision-making.

SAN DIEGO – A novel oral drug for inflammatory bowel disease showed good safety and efficacy data in preliminary clinical trial results.

Among a group of 32 patients with ulcerative colitis, the investigative drug ABX464 showed a decrease in Mayo score of over 50% and a drop in fecal calprotectin to near-normal levels. The safety profile was reassuring, and results were durable at the 9-month mark.

Coauthor Jean-Marc Steens, MD, presented results of the randomized, double-blind, placebo-controlled phase 2a study at the annual Digestive Disease Week^®, and noted that ABX464 is also being investigated as antiviral therapy for individuals with HIV/AIDS.

“Despite the major advances in the last 10 years with the introduction of biologics and [Janus kinase] inhibitors, there is still a huge unmet medical need for these patients,” said Dr. Steens, chief medical officer of Abivax (Paris), in an interview. “Large phase 3 studies with these recent drugs have shown that about two-thirds of the patients show a clinical response during induction, but that half of these responders will lose their response within the next 6-12 months. The safety profile of these drugs also includes severe infections, which is a major concern,” he said.

Dr. Steens presented the findings on behalf of first author Severine Vermeire, MD, chair of the department of chronic diseases, metabolism, and aging at Catholic University, Leuven, Belgium.

ABX464, a small-molecule oral medication, has been evaluated for safety among more than 180 patients with HIV as well as the patients with ulcerative colitis (UC) studied in the current trial. The drug increases expression of the microRNA precursor miR-124, with the result that “the inflammatory brake is applied,” explained Dr. Steens.

In the present study, whose primary outcome was safety, 23 patients with moderate to severe active UC were randomized to ABX464 50 mg once daily, and 9 to placebo. Patients were included if they had failed or were intolerant to immunomodulators, anti–tumor necrosis factor–alpha therapies, vedolizumab, or corticosteroids; the two groups had balanced disease and demographic characteristics. At baseline, patients had a total Mayo score of 6-12, and an endoscopic subscore of 2 or 3.

Three patients withdrew from the ABX464 arm by the end of 8 weeks: one because of adverse events (AEs), one withdrew consent, and the third declined to undergo endoscopy at the 8-week mark.

All treatment-emergent AEs were mild or moderate, with gastrointestinal disorders occurring in eight of the ABX464 patients and two placebo patients (34.8% and 22.2%, respectively.) Five ABX-464 patients (21.7%) experienced nervous system symptoms – mostly headaches, said Dr. Steens. No patients in the placebo arm had headache or other neurological AEs.

By the end of 8 weeks, 30% of the intention-to-treat ABX464 group was in clinical remission, compared with 11% of the placebo group; this was not a statistically significant difference (P = .16). The proportion of ABX464 patients who had a clinical response just missed statistical significance, compared with placebo (61% versus 33%; P = 06).

However, significant endoscopic improvement was seen in the ABX464 arm, with 43% having a Mayo endoscopy subscore of 0 or 1, compared with 11% in the placebo arm (P = .03).

The total Mayo score dropped by 53% in the ABX464 group, compared with 27% in the placebo group (P = .03); a partial Mayo score dropped by 62% for those in the active arm, compared with 32% in the placebo arm (P = .02).

“The major finding from the induction study was that all endpoints were going in the same direction in favor of ABX464, even reaching statistical significance for endoscopy as well, and total and partial Mayo score,” said Dr. Steens.

Patients underwent rectal biopsies at the end of 8 weeks, and miR-124 expression increased more than sevenfold from baseline for those taking ABX464, compared with a small increase in the placebo group (7.69- versus 1.46-fold; P = .004). Expression of miR-124 in total blood also increased – by over 800-fold – at study day 28 for the ABX464 arm. Levels were sustained at more than 700-fold at study day 56 in this group. Placebo arm participants saw an insignificant rise in miR-124 blood levels.

Dr. Steens reported that 22 patients, including 7 who had originally been placebo arm participants, continued into the maintenance phase of the study. Nineteen patients have now had a median of over 400 days of exposure to ABX464, with sustained significant improvement in partial Mayo scores from a baseline of 6 to scores below 2 at 6 and 9 months. Fecal calprotectin scores have dropped from a median 1,044 mcg/g at baseline to 23.5 mcg/g at 9 months.

Next steps include the 12-month assessment, which includes another endoscopy, said Dr. Steens. Also, a phase 2b study is seeking to enroll 232 patients who have moderate to severe ulcerative colitis, with room within the enrollment scheme for new study sites, said Dr. Steens. This larger study will have arms in which the current 50-mg oral dose is doubled and halved, as well as a placebo arm, he said. The medication will also be trialed for Crohn’s disease and rheumatoid arthritis.

The small sample size is an inherent limitation of this early-stage clinical trial, noted Dr. Steens.

Dr. Steens reported being an employee and holding shares in Abivax, which funded the study.

koakes@mdedge.com

SOURCE: Vermeire S et al. DDW 2019, Abstract 1007.

SAN DIEGO – A novel oral drug for inflammatory bowel disease showed good safety and efficacy data in preliminary clinical trial results.

Among a group of 32 patients with ulcerative colitis, the investigative drug ABX464 showed a decrease in Mayo score of over 50% and a drop in fecal calprotectin to near-normal levels. The safety profile was reassuring, and results were durable at the 9-month mark.

Coauthor Jean-Marc Steens, MD, presented results of the randomized, double-blind, placebo-controlled phase 2a study at the annual Digestive Disease Week^®, and noted that ABX464 is also being investigated as antiviral therapy for individuals with HIV/AIDS.

“Despite the major advances in the last 10 years with the introduction of biologics and [Janus kinase] inhibitors, there is still a huge unmet medical need for these patients,” said Dr. Steens, chief medical officer of Abivax (Paris), in an interview. “Large phase 3 studies with these recent drugs have shown that about two-thirds of the patients show a clinical response during induction, but that half of these responders will lose their response within the next 6-12 months. The safety profile of these drugs also includes severe infections, which is a major concern,” he said.

Dr. Steens presented the findings on behalf of first author Severine Vermeire, MD, chair of the department of chronic diseases, metabolism, and aging at Catholic University, Leuven, Belgium.

ABX464, a small-molecule oral medication, has been evaluated for safety among more than 180 patients with HIV as well as the patients with ulcerative colitis (UC) studied in the current trial. The drug increases expression of the microRNA precursor miR-124, with the result that “the inflammatory brake is applied,” explained Dr. Steens.

In the present study, whose primary outcome was safety, 23 patients with moderate to severe active UC were randomized to ABX464 50 mg once daily, and 9 to placebo. Patients were included if they had failed or were intolerant to immunomodulators, anti–tumor necrosis factor–alpha therapies, vedolizumab, or corticosteroids; the two groups had balanced disease and demographic characteristics. At baseline, patients had a total Mayo score of 6-12, and an endoscopic subscore of 2 or 3.

Three patients withdrew from the ABX464 arm by the end of 8 weeks: one because of adverse events (AEs), one withdrew consent, and the third declined to undergo endoscopy at the 8-week mark.

All treatment-emergent AEs were mild or moderate, with gastrointestinal disorders occurring in eight of the ABX464 patients and two placebo patients (34.8% and 22.2%, respectively.) Five ABX-464 patients (21.7%) experienced nervous system symptoms – mostly headaches, said Dr. Steens. No patients in the placebo arm had headache or other neurological AEs.

By the end of 8 weeks, 30% of the intention-to-treat ABX464 group was in clinical remission, compared with 11% of the placebo group; this was not a statistically significant difference (P = .16). The proportion of ABX464 patients who had a clinical response just missed statistical significance, compared with placebo (61% versus 33%; P = 06).

However, significant endoscopic improvement was seen in the ABX464 arm, with 43% having a Mayo endoscopy subscore of 0 or 1, compared with 11% in the placebo arm (P = .03).

The total Mayo score dropped by 53% in the ABX464 group, compared with 27% in the placebo group (P = .03); a partial Mayo score dropped by 62% for those in the active arm, compared with 32% in the placebo arm (P = .02).

“The major finding from the induction study was that all endpoints were going in the same direction in favor of ABX464, even reaching statistical significance for endoscopy as well, and total and partial Mayo score,” said Dr. Steens.

Patients underwent rectal biopsies at the end of 8 weeks, and miR-124 expression increased more than sevenfold from baseline for those taking ABX464, compared with a small increase in the placebo group (7.69- versus 1.46-fold; P = .004). Expression of miR-124 in total blood also increased – by over 800-fold – at study day 28 for the ABX464 arm. Levels were sustained at more than 700-fold at study day 56 in this group. Placebo arm participants saw an insignificant rise in miR-124 blood levels.

Dr. Steens reported that 22 patients, including 7 who had originally been placebo arm participants, continued into the maintenance phase of the study. Nineteen patients have now had a median of over 400 days of exposure to ABX464, with sustained significant improvement in partial Mayo scores from a baseline of 6 to scores below 2 at 6 and 9 months. Fecal calprotectin scores have dropped from a median 1,044 mcg/g at baseline to 23.5 mcg/g at 9 months.

Next steps include the 12-month assessment, which includes another endoscopy, said Dr. Steens. Also, a phase 2b study is seeking to enroll 232 patients who have moderate to severe ulcerative colitis, with room within the enrollment scheme for new study sites, said Dr. Steens. This larger study will have arms in which the current 50-mg oral dose is doubled and halved, as well as a placebo arm, he said. The medication will also be trialed for Crohn’s disease and rheumatoid arthritis.

The small sample size is an inherent limitation of this early-stage clinical trial, noted Dr. Steens.

Dr. Steens reported being an employee and holding shares in Abivax, which funded the study.

koakes@mdedge.com

SOURCE: Vermeire S et al. DDW 2019, Abstract 1007.

SAN DIEGO – A novel oral drug for inflammatory bowel disease showed good safety and efficacy data in preliminary clinical trial results.

Among a group of 32 patients with ulcerative colitis, the investigative drug ABX464 showed a decrease in Mayo score of over 50% and a drop in fecal calprotectin to near-normal levels. The safety profile was reassuring, and results were durable at the 9-month mark.

Coauthor Jean-Marc Steens, MD, presented results of the randomized, double-blind, placebo-controlled phase 2a study at the annual Digestive Disease Week^®, and noted that ABX464 is also being investigated as antiviral therapy for individuals with HIV/AIDS.

“Despite the major advances in the last 10 years with the introduction of biologics and [Janus kinase] inhibitors, there is still a huge unmet medical need for these patients,” said Dr. Steens, chief medical officer of Abivax (Paris), in an interview. “Large phase 3 studies with these recent drugs have shown that about two-thirds of the patients show a clinical response during induction, but that half of these responders will lose their response within the next 6-12 months. The safety profile of these drugs also includes severe infections, which is a major concern,” he said.

Dr. Steens presented the findings on behalf of first author Severine Vermeire, MD, chair of the department of chronic diseases, metabolism, and aging at Catholic University, Leuven, Belgium.

ABX464, a small-molecule oral medication, has been evaluated for safety among more than 180 patients with HIV as well as the patients with ulcerative colitis (UC) studied in the current trial. The drug increases expression of the microRNA precursor miR-124, with the result that “the inflammatory brake is applied,” explained Dr. Steens.

In the present study, whose primary outcome was safety, 23 patients with moderate to severe active UC were randomized to ABX464 50 mg once daily, and 9 to placebo. Patients were included if they had failed or were intolerant to immunomodulators, anti–tumor necrosis factor–alpha therapies, vedolizumab, or corticosteroids; the two groups had balanced disease and demographic characteristics. At baseline, patients had a total Mayo score of 6-12, and an endoscopic subscore of 2 or 3.

Three patients withdrew from the ABX464 arm by the end of 8 weeks: one because of adverse events (AEs), one withdrew consent, and the third declined to undergo endoscopy at the 8-week mark.

All treatment-emergent AEs were mild or moderate, with gastrointestinal disorders occurring in eight of the ABX464 patients and two placebo patients (34.8% and 22.2%, respectively.) Five ABX-464 patients (21.7%) experienced nervous system symptoms – mostly headaches, said Dr. Steens. No patients in the placebo arm had headache or other neurological AEs.

By the end of 8 weeks, 30% of the intention-to-treat ABX464 group was in clinical remission, compared with 11% of the placebo group; this was not a statistically significant difference (P = .16). The proportion of ABX464 patients who had a clinical response just missed statistical significance, compared with placebo (61% versus 33%; P = 06).

However, significant endoscopic improvement was seen in the ABX464 arm, with 43% having a Mayo endoscopy subscore of 0 or 1, compared with 11% in the placebo arm (P = .03).

The total Mayo score dropped by 53% in the ABX464 group, compared with 27% in the placebo group (P = .03); a partial Mayo score dropped by 62% for those in the active arm, compared with 32% in the placebo arm (P = .02).

“The major finding from the induction study was that all endpoints were going in the same direction in favor of ABX464, even reaching statistical significance for endoscopy as well, and total and partial Mayo score,” said Dr. Steens.

Patients underwent rectal biopsies at the end of 8 weeks, and miR-124 expression increased more than sevenfold from baseline for those taking ABX464, compared with a small increase in the placebo group (7.69- versus 1.46-fold; P = .004). Expression of miR-124 in total blood also increased – by over 800-fold – at study day 28 for the ABX464 arm. Levels were sustained at more than 700-fold at study day 56 in this group. Placebo arm participants saw an insignificant rise in miR-124 blood levels.

Dr. Steens reported that 22 patients, including 7 who had originally been placebo arm participants, continued into the maintenance phase of the study. Nineteen patients have now had a median of over 400 days of exposure to ABX464, with sustained significant improvement in partial Mayo scores from a baseline of 6 to scores below 2 at 6 and 9 months. Fecal calprotectin scores have dropped from a median 1,044 mcg/g at baseline to 23.5 mcg/g at 9 months.

Next steps include the 12-month assessment, which includes another endoscopy, said Dr. Steens. Also, a phase 2b study is seeking to enroll 232 patients who have moderate to severe ulcerative colitis, with room within the enrollment scheme for new study sites, said Dr. Steens. This larger study will have arms in which the current 50-mg oral dose is doubled and halved, as well as a placebo arm, he said. The medication will also be trialed for Crohn’s disease and rheumatoid arthritis.

The small sample size is an inherent limitation of this early-stage clinical trial, noted Dr. Steens.

Dr. Steens reported being an employee and holding shares in Abivax, which funded the study.

koakes@mdedge.com

SOURCE: Vermeire S et al. DDW 2019, Abstract 1007.

Inflammatory bowel disease (IBD) treatment remains a challenge in part because care is often fragmented among providers in different specialties, according to the American Gastroenterological Association. To address the need for provider coordination, the AGA has issued a new referral pathway for IBD care, published in Gastroenterology.

“The goal of this pathway is to offer guidance to primary care, emergency department, and gastroenterology providers, by helping identify patients at risk of or diagnosed with IBD and provide direction on initiating appropriate patient referrals,” wrote lead author Jami Kinnucan, MD, of the University of Michigan, Ann Arbor, and members of the AGA workgroup.

In particular, the pathway focuses on gaps in IBD care related to inflammatory issues, mental health, and nutrition. The work group included not only gastroenterologists, but also a primary care physician, mental/behavioral health specialist, registered dietitian/nutritionist, critical care specialist, nurse practitioner, physician group representative, and a patient advocacy representative.

The pathway identifies the top three areas where IBD patients usually present with symptoms: the emergency department, primary care office, and gastroenterology office.

The work group developed a list of key characteristics associated with increased morbidity, established IBD, or IBD-related complications that can be separated into high-risk, moderate-risk, and low-risk groups to help clinicians determine the timing of and need for referrals.

The pathway uses a sample patient presenting with GI symptoms such as bloody diarrhea; GI bleeding; anemia; fecal urgency; fever; abdominal pain; weight loss; and pain, swelling, or redness in the joints. Clinicians then apply the key characteristics to triage the patients into the risk groups.

High-risk characteristics include history of perianal or severe rectal disease, or deep ulcers in the GI mucosa; two or more emergency department visits for GI problems within the past 6 months, severe anemia, inadequate response to outpatient IBD therapy, history of IBD-related surgery, and malnourishment.

Moderate-risk characteristics include anemia without clinical symptoms, chronic corticosteroid use, and no emergency department or other GI medical visits within the past year.

Low-risk characteristics include chronic narcotic use, one or more comorbidities (such as heart failure, active hepatitis B, oncologic malignancy, lupus, GI infections, primary sclerosing cholangitis, viral hepatitis, and celiac disease), one or more relevant mental health conditions (such as depression, anxiety, or chronic pain), and nonadherence to IBD medical therapies.

“Referrals should be based on the highest level of risk present, in the event that a patient has characteristics that fall in more than one risk category,” the work group wrote.

To further guide clinicians in referring patients with possible or diagnosed IBD to gastroenterology specialists and to mental health and nutrition specialists, the work group developed an IBD Characteristics Assessment Checklist and a Referral Feedback form to accompany the pathway.

The checklist is designed for use by any health care professional to help identify whether a patient needs to be referred based on the key characteristics; the feedback form gives gastroenterologists a template to communicate with referring physicians about comanagement strategies for the patient.

The pathway also includes more details on how clinicians can tackle barriers to mental health and nutrition care for IBD patients.

“Until further evaluations are conducted, the work group encourages the immediate use of the pathway to begin addressing the needed improvements for IBD care coordination and communication between the different IBD providers,” the authors wrote.

Dr. Kinnucan disclosed serving as a consultant for AbbVie, Janssen, and Pfizer and serving on the Patient Education Committee of the Crohn’s and Colitis Foundation.

SOURCE: Kinnucan J et al. Gastroenterology. 2019. doi: 10.1053/j.gastro.2019.03.064.

Inflammatory bowel disease (IBD) treatment remains a challenge in part because care is often fragmented among providers in different specialties, according to the American Gastroenterological Association. To address the need for provider coordination, the AGA has issued a new referral pathway for IBD care, published in Gastroenterology.

“The goal of this pathway is to offer guidance to primary care, emergency department, and gastroenterology providers, by helping identify patients at risk of or diagnosed with IBD and provide direction on initiating appropriate patient referrals,” wrote lead author Jami Kinnucan, MD, of the University of Michigan, Ann Arbor, and members of the AGA workgroup.

In particular, the pathway focuses on gaps in IBD care related to inflammatory issues, mental health, and nutrition. The work group included not only gastroenterologists, but also a primary care physician, mental/behavioral health specialist, registered dietitian/nutritionist, critical care specialist, nurse practitioner, physician group representative, and a patient advocacy representative.

The pathway identifies the top three areas where IBD patients usually present with symptoms: the emergency department, primary care office, and gastroenterology office.

The work group developed a list of key characteristics associated with increased morbidity, established IBD, or IBD-related complications that can be separated into high-risk, moderate-risk, and low-risk groups to help clinicians determine the timing of and need for referrals.

The pathway uses a sample patient presenting with GI symptoms such as bloody diarrhea; GI bleeding; anemia; fecal urgency; fever; abdominal pain; weight loss; and pain, swelling, or redness in the joints. Clinicians then apply the key characteristics to triage the patients into the risk groups.

High-risk characteristics include history of perianal or severe rectal disease, or deep ulcers in the GI mucosa; two or more emergency department visits for GI problems within the past 6 months, severe anemia, inadequate response to outpatient IBD therapy, history of IBD-related surgery, and malnourishment.

Moderate-risk characteristics include anemia without clinical symptoms, chronic corticosteroid use, and no emergency department or other GI medical visits within the past year.

Low-risk characteristics include chronic narcotic use, one or more comorbidities (such as heart failure, active hepatitis B, oncologic malignancy, lupus, GI infections, primary sclerosing cholangitis, viral hepatitis, and celiac disease), one or more relevant mental health conditions (such as depression, anxiety, or chronic pain), and nonadherence to IBD medical therapies.

“Referrals should be based on the highest level of risk present, in the event that a patient has characteristics that fall in more than one risk category,” the work group wrote.

To further guide clinicians in referring patients with possible or diagnosed IBD to gastroenterology specialists and to mental health and nutrition specialists, the work group developed an IBD Characteristics Assessment Checklist and a Referral Feedback form to accompany the pathway.

The checklist is designed for use by any health care professional to help identify whether a patient needs to be referred based on the key characteristics; the feedback form gives gastroenterologists a template to communicate with referring physicians about comanagement strategies for the patient.

The pathway also includes more details on how clinicians can tackle barriers to mental health and nutrition care for IBD patients.

“Until further evaluations are conducted, the work group encourages the immediate use of the pathway to begin addressing the needed improvements for IBD care coordination and communication between the different IBD providers,” the authors wrote.

Dr. Kinnucan disclosed serving as a consultant for AbbVie, Janssen, and Pfizer and serving on the Patient Education Committee of the Crohn’s and Colitis Foundation.

SOURCE: Kinnucan J et al. Gastroenterology. 2019. doi: 10.1053/j.gastro.2019.03.064.

Inflammatory bowel disease (IBD) treatment remains a challenge in part because care is often fragmented among providers in different specialties, according to the American Gastroenterological Association. To address the need for provider coordination, the AGA has issued a new referral pathway for IBD care, published in Gastroenterology.

“The goal of this pathway is to offer guidance to primary care, emergency department, and gastroenterology providers, by helping identify patients at risk of or diagnosed with IBD and provide direction on initiating appropriate patient referrals,” wrote lead author Jami Kinnucan, MD, of the University of Michigan, Ann Arbor, and members of the AGA workgroup.

In particular, the pathway focuses on gaps in IBD care related to inflammatory issues, mental health, and nutrition. The work group included not only gastroenterologists, but also a primary care physician, mental/behavioral health specialist, registered dietitian/nutritionist, critical care specialist, nurse practitioner, physician group representative, and a patient advocacy representative.

The pathway identifies the top three areas where IBD patients usually present with symptoms: the emergency department, primary care office, and gastroenterology office.

The work group developed a list of key characteristics associated with increased morbidity, established IBD, or IBD-related complications that can be separated into high-risk, moderate-risk, and low-risk groups to help clinicians determine the timing of and need for referrals.

The pathway uses a sample patient presenting with GI symptoms such as bloody diarrhea; GI bleeding; anemia; fecal urgency; fever; abdominal pain; weight loss; and pain, swelling, or redness in the joints. Clinicians then apply the key characteristics to triage the patients into the risk groups.

High-risk characteristics include history of perianal or severe rectal disease, or deep ulcers in the GI mucosa; two or more emergency department visits for GI problems within the past 6 months, severe anemia, inadequate response to outpatient IBD therapy, history of IBD-related surgery, and malnourishment.

Moderate-risk characteristics include anemia without clinical symptoms, chronic corticosteroid use, and no emergency department or other GI medical visits within the past year.

Low-risk characteristics include chronic narcotic use, one or more comorbidities (such as heart failure, active hepatitis B, oncologic malignancy, lupus, GI infections, primary sclerosing cholangitis, viral hepatitis, and celiac disease), one or more relevant mental health conditions (such as depression, anxiety, or chronic pain), and nonadherence to IBD medical therapies.

“Referrals should be based on the highest level of risk present, in the event that a patient has characteristics that fall in more than one risk category,” the work group wrote.

To further guide clinicians in referring patients with possible or diagnosed IBD to gastroenterology specialists and to mental health and nutrition specialists, the work group developed an IBD Characteristics Assessment Checklist and a Referral Feedback form to accompany the pathway.

The checklist is designed for use by any health care professional to help identify whether a patient needs to be referred based on the key characteristics; the feedback form gives gastroenterologists a template to communicate with referring physicians about comanagement strategies for the patient.

The pathway also includes more details on how clinicians can tackle barriers to mental health and nutrition care for IBD patients.

“Until further evaluations are conducted, the work group encourages the immediate use of the pathway to begin addressing the needed improvements for IBD care coordination and communication between the different IBD providers,” the authors wrote.

Dr. Kinnucan disclosed serving as a consultant for AbbVie, Janssen, and Pfizer and serving on the Patient Education Committee of the Crohn’s and Colitis Foundation.

SOURCE: Kinnucan J et al. Gastroenterology. 2019. doi: 10.1053/j.gastro.2019.03.064.

SAN FRANCISCO – Patients with alcohol use disorder reported a substantial decrease in drinking days, drinks per drinking day, and cravings in an ongoing trial of psilocybin at New York University.

M. Alexander Otto/MDedge News

“If this keeps going the way it looks like it’s going, I think it will lead to a large phase 3 trial that could be part of getting psilocybin rescheduled” from a schedule I drug, said NYU psychiatrist and lead investigator Kelley O’Donnell, MD, PhD.

The work builds on positive results from the 1950s and 1960s of LSD for alcoholism, before LSD research was largely abandoned. Researchers such as Dr. O’Donnell are revisiting the approach, but with psilocybin because, among other reasons, it has less stigma and a shorter duration that allows for outpatient use (J Psychopharmacol. 2012 Jul;26[7]:994-1002). The Drug Enforcement Administration currently classifies psilocybin, the psychoactive ingredient in hallucinogenic, or “magic” mushrooms, as schedule I. The results found by Dr. O’Donnell’s team and other factors, such as the low risk of abuse tied to the use of psilocybin, are leading some researchers to suggest that the drug should be reclassified to “no more restrictively than schedule IV” (Neuropsychopharm. 2018 Nov;142:143-66).

Dr. O’Donnell’s presentation was part of a recurring theme at the annual meeting of the American Psychiatric Association – the transformation of what were once considered street drugs into therapeutic tools. Favorable results also were reported for 3,4-Methylenedioxymethamphetamine (MDMA), commonly known as ecstasy, for posttraumatic stress disorder; ketamine for depression and suicidality; and marijuana for pain and other problems.

Concerning psilocybin for alcohol use disorder (AUD), Dr. O’Donnell said: “Some people have really profound psychological experiences that shift the way they think about themselves and the way alcohol is affecting their relationships. Therapy can work with that shift in meaning [to create] lasting change.”

Many people “say that they get exactly what they needed.” Sometimes, patients revisit a past trauma but with a greater openness and flexibility – and a growing sense of peace. “They access affective states they just don’t have access to normally,” she said.

Dr. O’Donnell recalled a man who said, after his trip: “Okay, I’m done. I’ve seen my true place in the world, and alcohol is interfering with my ability to take that place. I don’t need it, anymore.”

In another case, a woman hallucinated that she was sitting on a throne ascending through the universe, surrounded by the faces and voices of people she knew telling her she is a valuable and worthwhile person, and could take her place within the center of her universe without the sense of defectiveness and shame that often triggered her drinking. Happy little beer bottles told her: “We don’t need to be the enemy. We don’t need to be a part of your life,” Dr. O’Donnell said.

She and her team are pitting psilocybin against diphenhydramine as a control in the NYU AUD trial.

People are randomized, then undergo therapy focused equally on their alcohol use and preparing them for the drug experience. At week 4, they take their study medication – either 25 mg per 70 kg psilocybin or 50 mg diphenhydramine ­– in a relaxed living room–like setting, with classical or world music in the background. The study team avoids music with words in English. Two therapists, usually a man and a woman, are there as guides. The experience lasts a few hours; patients are debriefed afterward.

Patients undergo another round of counseling to understand the meaning of the experience, followed by a second dose, either 40 mg per 70 kg psilocybin or 100 mg diphenhydramine, at week 8. Patients are debriefed again and undergo a third month of counseling.

The results have not yet been unblinded, but Dr. O’Donnell and her team did find that, among their first 56 subjects, more intense mystical experiences, as gauged by the self-reported Mystical Experience Questionnaire (MEQ), correlated with greater treatment success.

Patients fill out the MEQ 8 hours after their dose, rating dimensions such as ego dissolution, oceanic boundlessness, joy, compassion, and openness. The maximum score is 1, the lowest 0, meaning no mystical effects. The median score among the 56 subjects was 0.26. The 30 or so patients who scored at or above that mark after their first medication session – as a group, their mean first MEQ score was 0.65 – reported a smaller percentage of drinking days at week 12 than those who scored below 0.26 (19% vs. 40%; P less than .05), with fewer drinks per drinking day (2.63 vs. 7.01; P less than .01); and lower craving (8.43 vs. 13.86 points on 30-point Penn Alcohol Craving Scale, P less than .01).

The groups were evenly matched at baseline. Both reported drinking an average of 3 out of 4 days, with an mean of 7.5 drinks per drinking day and a craving score of about 18. No differences were found in anxiety and depression scores, which were minimal in both groups.

More than half the subjects were men; the mean age was 46; and subjects were fairly well educated, reporting an average of 17 school years.

Dr. O’Donnell said she’s seen a range of experiences on psilocybin, but that bad trips are rare. Benzodiazepines are kept on hand, however, to help people who get too anxious, and an atypical antipsychotic is on hand to reverse hallucinatory effects.

Her team hopes to enroll 100 subjects and plans for follow-up past 12 weeks. Both Denver and Oakland, Calif., recently decriminalized psilocybin.

The work is being funded by the Heffter Research Institute. Dr. O’Donnell had no disclosures.

aotto@mdedge.com

SAN FRANCISCO – Patients with alcohol use disorder reported a substantial decrease in drinking days, drinks per drinking day, and cravings in an ongoing trial of psilocybin at New York University.

M. Alexander Otto/MDedge News

“If this keeps going the way it looks like it’s going, I think it will lead to a large phase 3 trial that could be part of getting psilocybin rescheduled” from a schedule I drug, said NYU psychiatrist and lead investigator Kelley O’Donnell, MD, PhD.

The work builds on positive results from the 1950s and 1960s of LSD for alcoholism, before LSD research was largely abandoned. Researchers such as Dr. O’Donnell are revisiting the approach, but with psilocybin because, among other reasons, it has less stigma and a shorter duration that allows for outpatient use (J Psychopharmacol. 2012 Jul;26[7]:994-1002). The Drug Enforcement Administration currently classifies psilocybin, the psychoactive ingredient in hallucinogenic, or “magic” mushrooms, as schedule I. The results found by Dr. O’Donnell’s team and other factors, such as the low risk of abuse tied to the use of psilocybin, are leading some researchers to suggest that the drug should be reclassified to “no more restrictively than schedule IV” (Neuropsychopharm. 2018 Nov;142:143-66).

Dr. O’Donnell’s presentation was part of a recurring theme at the annual meeting of the American Psychiatric Association – the transformation of what were once considered street drugs into therapeutic tools. Favorable results also were reported for 3,4-Methylenedioxymethamphetamine (MDMA), commonly known as ecstasy, for posttraumatic stress disorder; ketamine for depression and suicidality; and marijuana for pain and other problems.

Concerning psilocybin for alcohol use disorder (AUD), Dr. O’Donnell said: “Some people have really profound psychological experiences that shift the way they think about themselves and the way alcohol is affecting their relationships. Therapy can work with that shift in meaning [to create] lasting change.”

Many people “say that they get exactly what they needed.” Sometimes, patients revisit a past trauma but with a greater openness and flexibility – and a growing sense of peace. “They access affective states they just don’t have access to normally,” she said.

Dr. O’Donnell recalled a man who said, after his trip: “Okay, I’m done. I’ve seen my true place in the world, and alcohol is interfering with my ability to take that place. I don’t need it, anymore.”

In another case, a woman hallucinated that she was sitting on a throne ascending through the universe, surrounded by the faces and voices of people she knew telling her she is a valuable and worthwhile person, and could take her place within the center of her universe without the sense of defectiveness and shame that often triggered her drinking. Happy little beer bottles told her: “We don’t need to be the enemy. We don’t need to be a part of your life,” Dr. O’Donnell said.

She and her team are pitting psilocybin against diphenhydramine as a control in the NYU AUD trial.

People are randomized, then undergo therapy focused equally on their alcohol use and preparing them for the drug experience. At week 4, they take their study medication – either 25 mg per 70 kg psilocybin or 50 mg diphenhydramine ­– in a relaxed living room–like setting, with classical or world music in the background. The study team avoids music with words in English. Two therapists, usually a man and a woman, are there as guides. The experience lasts a few hours; patients are debriefed afterward.

Patients undergo another round of counseling to understand the meaning of the experience, followed by a second dose, either 40 mg per 70 kg psilocybin or 100 mg diphenhydramine, at week 8. Patients are debriefed again and undergo a third month of counseling.

The results have not yet been unblinded, but Dr. O’Donnell and her team did find that, among their first 56 subjects, more intense mystical experiences, as gauged by the self-reported Mystical Experience Questionnaire (MEQ), correlated with greater treatment success.

Patients fill out the MEQ 8 hours after their dose, rating dimensions such as ego dissolution, oceanic boundlessness, joy, compassion, and openness. The maximum score is 1, the lowest 0, meaning no mystical effects. The median score among the 56 subjects was 0.26. The 30 or so patients who scored at or above that mark after their first medication session – as a group, their mean first MEQ score was 0.65 – reported a smaller percentage of drinking days at week 12 than those who scored below 0.26 (19% vs. 40%; P less than .05), with fewer drinks per drinking day (2.63 vs. 7.01; P less than .01); and lower craving (8.43 vs. 13.86 points on 30-point Penn Alcohol Craving Scale, P less than .01).

The groups were evenly matched at baseline. Both reported drinking an average of 3 out of 4 days, with an mean of 7.5 drinks per drinking day and a craving score of about 18. No differences were found in anxiety and depression scores, which were minimal in both groups.

More than half the subjects were men; the mean age was 46; and subjects were fairly well educated, reporting an average of 17 school years.

Dr. O’Donnell said she’s seen a range of experiences on psilocybin, but that bad trips are rare. Benzodiazepines are kept on hand, however, to help people who get too anxious, and an atypical antipsychotic is on hand to reverse hallucinatory effects.

Her team hopes to enroll 100 subjects and plans for follow-up past 12 weeks. Both Denver and Oakland, Calif., recently decriminalized psilocybin.

The work is being funded by the Heffter Research Institute. Dr. O’Donnell had no disclosures.

aotto@mdedge.com

SAN FRANCISCO – Patients with alcohol use disorder reported a substantial decrease in drinking days, drinks per drinking day, and cravings in an ongoing trial of psilocybin at New York University.

M. Alexander Otto/MDedge News

“If this keeps going the way it looks like it’s going, I think it will lead to a large phase 3 trial that could be part of getting psilocybin rescheduled” from a schedule I drug, said NYU psychiatrist and lead investigator Kelley O’Donnell, MD, PhD.

The work builds on positive results from the 1950s and 1960s of LSD for alcoholism, before LSD research was largely abandoned. Researchers such as Dr. O’Donnell are revisiting the approach, but with psilocybin because, among other reasons, it has less stigma and a shorter duration that allows for outpatient use (J Psychopharmacol. 2012 Jul;26[7]:994-1002). The Drug Enforcement Administration currently classifies psilocybin, the psychoactive ingredient in hallucinogenic, or “magic” mushrooms, as schedule I. The results found by Dr. O’Donnell’s team and other factors, such as the low risk of abuse tied to the use of psilocybin, are leading some researchers to suggest that the drug should be reclassified to “no more restrictively than schedule IV” (Neuropsychopharm. 2018 Nov;142:143-66).

Dr. O’Donnell’s presentation was part of a recurring theme at the annual meeting of the American Psychiatric Association – the transformation of what were once considered street drugs into therapeutic tools. Favorable results also were reported for 3,4-Methylenedioxymethamphetamine (MDMA), commonly known as ecstasy, for posttraumatic stress disorder; ketamine for depression and suicidality; and marijuana for pain and other problems.

Concerning psilocybin for alcohol use disorder (AUD), Dr. O’Donnell said: “Some people have really profound psychological experiences that shift the way they think about themselves and the way alcohol is affecting their relationships. Therapy can work with that shift in meaning [to create] lasting change.”

Many people “say that they get exactly what they needed.” Sometimes, patients revisit a past trauma but with a greater openness and flexibility – and a growing sense of peace. “They access affective states they just don’t have access to normally,” she said.

Dr. O’Donnell recalled a man who said, after his trip: “Okay, I’m done. I’ve seen my true place in the world, and alcohol is interfering with my ability to take that place. I don’t need it, anymore.”

In another case, a woman hallucinated that she was sitting on a throne ascending through the universe, surrounded by the faces and voices of people she knew telling her she is a valuable and worthwhile person, and could take her place within the center of her universe without the sense of defectiveness and shame that often triggered her drinking. Happy little beer bottles told her: “We don’t need to be the enemy. We don’t need to be a part of your life,” Dr. O’Donnell said.

She and her team are pitting psilocybin against diphenhydramine as a control in the NYU AUD trial.

People are randomized, then undergo therapy focused equally on their alcohol use and preparing them for the drug experience. At week 4, they take their study medication – either 25 mg per 70 kg psilocybin or 50 mg diphenhydramine ­– in a relaxed living room–like setting, with classical or world music in the background. The study team avoids music with words in English. Two therapists, usually a man and a woman, are there as guides. The experience lasts a few hours; patients are debriefed afterward.

Patients undergo another round of counseling to understand the meaning of the experience, followed by a second dose, either 40 mg per 70 kg psilocybin or 100 mg diphenhydramine, at week 8. Patients are debriefed again and undergo a third month of counseling.

The results have not yet been unblinded, but Dr. O’Donnell and her team did find that, among their first 56 subjects, more intense mystical experiences, as gauged by the self-reported Mystical Experience Questionnaire (MEQ), correlated with greater treatment success.

Patients fill out the MEQ 8 hours after their dose, rating dimensions such as ego dissolution, oceanic boundlessness, joy, compassion, and openness. The maximum score is 1, the lowest 0, meaning no mystical effects. The median score among the 56 subjects was 0.26. The 30 or so patients who scored at or above that mark after their first medication session – as a group, their mean first MEQ score was 0.65 – reported a smaller percentage of drinking days at week 12 than those who scored below 0.26 (19% vs. 40%; P less than .05), with fewer drinks per drinking day (2.63 vs. 7.01; P less than .01); and lower craving (8.43 vs. 13.86 points on 30-point Penn Alcohol Craving Scale, P less than .01).

The groups were evenly matched at baseline. Both reported drinking an average of 3 out of 4 days, with an mean of 7.5 drinks per drinking day and a craving score of about 18. No differences were found in anxiety and depression scores, which were minimal in both groups.

More than half the subjects were men; the mean age was 46; and subjects were fairly well educated, reporting an average of 17 school years.

Dr. O’Donnell said she’s seen a range of experiences on psilocybin, but that bad trips are rare. Benzodiazepines are kept on hand, however, to help people who get too anxious, and an atypical antipsychotic is on hand to reverse hallucinatory effects.

Her team hopes to enroll 100 subjects and plans for follow-up past 12 weeks. Both Denver and Oakland, Calif., recently decriminalized psilocybin.

The work is being funded by the Heffter Research Institute. Dr. O’Donnell had no disclosures.

aotto@mdedge.com

Measles cases in the United States have topped 1,000 for the first time since 1992, according to the Centers for Disease Control and Prevention.

The 41 new cases reported for the week ending June 6 bring the total for the year to 1,022, the CDC reported June 10, and that is more than any year since 1992, when there were 2,237 cases.

Idaho and Virginia reported their first cases of 2019, which makes a total of 28 states with measles cases this year. The Idaho case was reported in Latah County and is the state’s first since 2001. In Virginia, health officials are investigating possible contacts with an infected individual at Dulles International Airport and two other locations on June 2 and 4.

Outbreaks in Georgia, Maryland, and Michigan have ended, while seven others continue in California (Butte, Los Angeles, and Sacramento Counties), New York (Rockland County and New York City), Pennsylvania, and Washington, the CDC said. New York City has the largest outbreak this year with 509 cases through June 3, most of them occurring in Brooklyn.

rfranki@mdedge.com

Measles cases in the United States have topped 1,000 for the first time since 1992, according to the Centers for Disease Control and Prevention.

The 41 new cases reported for the week ending June 6 bring the total for the year to 1,022, the CDC reported June 10, and that is more than any year since 1992, when there were 2,237 cases.

Idaho and Virginia reported their first cases of 2019, which makes a total of 28 states with measles cases this year. The Idaho case was reported in Latah County and is the state’s first since 2001. In Virginia, health officials are investigating possible contacts with an infected individual at Dulles International Airport and two other locations on June 2 and 4.

Outbreaks in Georgia, Maryland, and Michigan have ended, while seven others continue in California (Butte, Los Angeles, and Sacramento Counties), New York (Rockland County and New York City), Pennsylvania, and Washington, the CDC said. New York City has the largest outbreak this year with 509 cases through June 3, most of them occurring in Brooklyn.

rfranki@mdedge.com

Measles cases in the United States have topped 1,000 for the first time since 1992, according to the Centers for Disease Control and Prevention.

The 41 new cases reported for the week ending June 6 bring the total for the year to 1,022, the CDC reported June 10, and that is more than any year since 1992, when there were 2,237 cases.

Idaho and Virginia reported their first cases of 2019, which makes a total of 28 states with measles cases this year. The Idaho case was reported in Latah County and is the state’s first since 2001. In Virginia, health officials are investigating possible contacts with an infected individual at Dulles International Airport and two other locations on June 2 and 4.

Outbreaks in Georgia, Maryland, and Michigan have ended, while seven others continue in California (Butte, Los Angeles, and Sacramento Counties), New York (Rockland County and New York City), Pennsylvania, and Washington, the CDC said. New York City has the largest outbreak this year with 509 cases through June 3, most of them occurring in Brooklyn.

rfranki@mdedge.com

The Food and Drug Administration has granted accelerated approval to Polivy (polatuzumab vedotin-piiq), in conjunction with bendamustine and a rituximab product, for the treatment of adult patients with relapsed or refractory diffuse large B-cell lymphoma (DLBCL) who have undergone at least two prior therapies.

The FDA approval is based on results of an open-label, multicenter clinical trial of 80 patients with DLBCL who had undergone at least one prior regimen. Patients received either Polivy plus bendamustine and rituximab or only bendamustine and rituximab for six 21-day cycles. At the completion of therapy, 40% of patients who received Polivy in conjunction with bendamustine and rituximab achieved a complete response, compared with 18% of patients who received only bendamustine and rituximab; total response was 63% in those who received Polivy and 25% in those who did not.

The most common adverse events included neutropenia, thrombocytopenia, anemia, peripheral neuropathy, fatigue, diarrhea, pyrexia, decreased appetite, and pneumonia. Serious adverse events occurred in 64% of patients, most commonly from infection; the most common cause for treatment discontinuation was cytopenia.

The recommended dose of Polivy is 1.8 mg/kg as an intravenous infusion over 90 minutes every 21 days for six cycles in combination with bendamustine and a rituximab product, the FDA said. Subsequent infusions may be administered over 30 minutes if the previous infusion is tolerated.

Find the full press release on the FDA website.

lfranki@mdedge.com

The Food and Drug Administration has granted accelerated approval to Polivy (polatuzumab vedotin-piiq), in conjunction with bendamustine and a rituximab product, for the treatment of adult patients with relapsed or refractory diffuse large B-cell lymphoma (DLBCL) who have undergone at least two prior therapies.

The FDA approval is based on results of an open-label, multicenter clinical trial of 80 patients with DLBCL who had undergone at least one prior regimen. Patients received either Polivy plus bendamustine and rituximab or only bendamustine and rituximab for six 21-day cycles. At the completion of therapy, 40% of patients who received Polivy in conjunction with bendamustine and rituximab achieved a complete response, compared with 18% of patients who received only bendamustine and rituximab; total response was 63% in those who received Polivy and 25% in those who did not.

The most common adverse events included neutropenia, thrombocytopenia, anemia, peripheral neuropathy, fatigue, diarrhea, pyrexia, decreased appetite, and pneumonia. Serious adverse events occurred in 64% of patients, most commonly from infection; the most common cause for treatment discontinuation was cytopenia.

The recommended dose of Polivy is 1.8 mg/kg as an intravenous infusion over 90 minutes every 21 days for six cycles in combination with bendamustine and a rituximab product, the FDA said. Subsequent infusions may be administered over 30 minutes if the previous infusion is tolerated.

Find the full press release on the FDA website.

lfranki@mdedge.com

The Food and Drug Administration has granted accelerated approval to Polivy (polatuzumab vedotin-piiq), in conjunction with bendamustine and a rituximab product, for the treatment of adult patients with relapsed or refractory diffuse large B-cell lymphoma (DLBCL) who have undergone at least two prior therapies.

The FDA approval is based on results of an open-label, multicenter clinical trial of 80 patients with DLBCL who had undergone at least one prior regimen. Patients received either Polivy plus bendamustine and rituximab or only bendamustine and rituximab for six 21-day cycles. At the completion of therapy, 40% of patients who received Polivy in conjunction with bendamustine and rituximab achieved a complete response, compared with 18% of patients who received only bendamustine and rituximab; total response was 63% in those who received Polivy and 25% in those who did not.

The most common adverse events included neutropenia, thrombocytopenia, anemia, peripheral neuropathy, fatigue, diarrhea, pyrexia, decreased appetite, and pneumonia. Serious adverse events occurred in 64% of patients, most commonly from infection; the most common cause for treatment discontinuation was cytopenia.

The recommended dose of Polivy is 1.8 mg/kg as an intravenous infusion over 90 minutes every 21 days for six cycles in combination with bendamustine and a rituximab product, the FDA said. Subsequent infusions may be administered over 30 minutes if the previous infusion is tolerated.

Find the full press release on the FDA website.

lfranki@mdedge.com