Gabriel J. Escobar, MD

Changed

Mon, 01/30/2017 - 11:14

Display Headline

Early detection of critical illness outside the intensive care unit: Clarifying treatment plans and honoring goals of care using a supportive care team

Author(s)

Zoe Sutton, RN, MSN, PNP

The likelihood of meaningful survival after cardiopulmonary arrest is low and even lower the longer the patient has been in the hospital[1, 2]; realization of this[3] played a major role in the development of rapid response teams (RRTs).[4] As noted elsewhere in this journal, the limited success of these teams[5, 6, 7] has inspired efforts to develop systems to identify patients at risk of deterioration much earlier.

Whereas a number of recent reports have described end‐of‐life care issues in the context of RRT operations,[8, 9, 10, 11, 12, 13, 14, 15, 16] descriptions of how one might incorporate respecting patient preferences into development of a response arm, particularly one meant to scale up to a multiple hospital system, are largely absent from the literature. In this article, we describe the implementation process for integrating palliative care and the honoring of patient choices, which we refer to as supportive care, with an automated early warning system (EWS) and an RRT.

The context of this work is a pilot project conducted at 2 community hospitals, the Kaiser Permanente Northern California (KPNC) Sacramento (200 beds) and South San Francisco (100 beds) medical centers. Our focus was to develop an approach that could serve as the basis for future dissemination to the remaining 19 KPNC hospitals, regardless of their size. Our work incorporated the Respecting Choices model,[17] which has been endorsed by KPNC for all its hospitals and clinics. We describe the workflow we developed to embed the supportive care team's (SCT) reactive and proactive components into the EWS response arm. We also provide a granular description of how our approach worked in practice, as evidenced by the combined patient and provider experiences captured in 5 vignettes as well as some preliminary data obtained by chart review

When patients arrive in the hospital, they may or may not have had a discussion about their care escalation and resuscitation preferences. As noted by Escobar and Dellinger[18] elsewhere in this issue of the Journal of Hospital Medicine, patients with documented restricted resuscitation preferences (eg, do not resuscitate [DNR] or partial code) at the time of admission to the hospital account for slightly more than half of the hospital deaths at 30 days after admission. In general, these stated preferences are honored.

Significant proportions of patients are unstable at the time of admission or have a significant underlying chronic illness burden predisposing them to unexpected deterioration. Often these patients lose decision‐making capacity when their condition worsens. We need to ensure we honor their wishes and identify the correct surrogate.

To make sure a patient's wishes are clear, we developed a workflow that included 2 components. One component is meant to ensure that patient preferences are honored following a EWS alert. This allows for contingencies, including the likelihood that a physician will not be available to discuss patient wishes due to clinical demands. Although it may appear that the role of the hospitalist is supplanted, in fact this is not the case. The only person who has authority to change a patient's code status is the hospitalist, and they always talk to the patient or their surrogate. The purpose of the teams described in this report is to provide backup, particularly in those instances where the hospitalist is tied up elsewhere (eg, the emergency department). Our workflows also facilitate the integration of the clinical with the palliative care response. The other component employs the EWS's ancillary elements (provision of a severity of illness score and longitudinal comorbidity score in real time) to screen patients who might need the SCT. This allows us to identify patients who are at high risk for deterioration in whom an alert has not yet been issued due to acute instability or comorbid burden (leading to high probability of unexpected deterioration) or both and who do not have stated goals of care and/or an identified surrogate.

IMPLEMENTATION APPROACH

We developed our workflow using the Institute for Healthcare Improvement's Plan‐Do‐Study‐Act approach.[19, 20] Our first finding was that most alerts did not require a rapid intervention by the SCT. Both sites reserved time in the SCT physicians' schedule and considered changing staffing levels (the smaller site only had funding for physician support 20 hours per week), but neither had to make such changes. One reason for this was that we increased social worker availability, particularly for off hours (to cover the contingency where an alert was issued in the middle of the night while the on‐call hospitalist was handling an admission in the emergency department). The second was that, as is described by Escobar et al.,[21] the EWS provides a risk of deterioration in the next 12 hours (as opposed to a code blue or regular RRT call, which indicate the need for immediate action) and provides an opportunity for spending time with patients without the constraints of an ongoing resuscitation.

We also found that of the patients who triggered an alert, approximately half would have been flagged for a palliative care referral using our own internal screening tool. Furthermore, having longitudinal comorbidity (Comorbidity Point Score, version 2 [COPS2]) and severity of illness (Laboratory‐Based Acute Physiology Score, version 2) scores[22] facilitated the identification of patients who needed review of their preferences with respect to escalation of care. Currently, our primary case‐finding criterion for proactive SCT consultation is a COPS2 >65, which is associated with a 10.8%, 30‐day mortality risk. Overall, the SCT was asked to see about 25% of patients in whom an alert was triggered.

The workflows we developed were employed at the first site to go live (South San Francisco, 7000 annual discharges, Figure 1) and then modified at Sacramento (14,000 annual discharges, Figure 2). Because the hospitals differ in several respects, from size and patient population to staffing, the workflows are slightly different.

Workflow for integrating Respecting Choices model with a real‐time early warning system at Kaiser Permanente South San Francisco. See text for additional details. Abbreviations: EWS, early warning system, EMR, electronic medical record; pt, patient; LCP, life care planning; HBS, hospital based specialist; RN, registered nurse; RRT, rapid response team; SCT, supportive care team; SW, social worker.

Workflow for integrating Respecting Choices model with a real‐time early warning system at Kaiser Permanente Sacramento. See text for additional details. Abbreviations: EWS, early warning system, EMR, electronic medical record; RN, registered nurse; RRT, rapid response team; SCT, supportive care team.

The EWS provides deterioration probabilities every 6 hours, and first responders (RRT nurses) intervene when this probability is 8%. The RRT nurse can activate the clinical response arm, the Respecting Choices pathway, or both. In South San Francisco, which lacked the resources to staff supportive care 24 hours a day/7 days a week, the RRT contacts a medical social worker (MSW) who performs an immediate record review. If this identifies something meriting urgent communication (eg, conflicting or absent information regarding a patient's surrogate), the MSW alerts the hospitalist. The MSW documents findings and ensures that a regular MSW consult occurs the next day. If the MSW feels the patient needs an SCT consult, the MSW alerts the team (this does not preclude a hospitalist or RRT nurse from initiating SCT consultation). At the Sacramento site, where the SCT team is staffed 24 hours a day/7 days a week, it is possible to bypass the MSW step. In addition, each morning the SCT reviews all alerts issued during the previous 24 hours to determine if an SCT consult is needed. In addition, the SCT also proactively reviews the COPS2 scores on all admissions to identify patients who could benefit from an SCT consult. Although surrogate identification and clarifying goals of care are essential, the SCT also helps patients in other ways, as is evident from the following case studies.

The major difference between the palliative care team and the SCT is that the SCT includes the inpatient social worker as part of the team. The SCT has a more focused role (its efforts center on aligning patient goals and priorities with the care that will actually be provided). In contrast, the palliative care team has other functions (eg, pain and symptom management) that are not necessarily associated with life care planning or the alert response.

Considerable overlap exists between patients who trigger an alert and those who would have met screening criteria established prior to EWS deployment. Although this is evolving, we can say that, in general, both sites are moving to an or criterion for involving the SCT (patient meets traditional criteria of the screening tool or triggers alert). Further, as KPNC begins adding more sites to the system, serious consideration is being given to only employing the COPS2 score as the primary screening criterion.

CASE STUDY 1: SURROGATE IDENTIFICATION

Mr. Smith, a 78‐year‐old man with congestive heart failure (CHF), atrial fibrillation, severe chronic obstructive pulmonary disease, and history of stroke, was admitted due to CHF exacerbation. The morning after admission, he experienced uncontrolled tachycardia associated with low oxygen saturation, triggering an alert. The hospitalist stabilized him and documented the treatment plan as follows: If worsening signs (shortness of breath/wheezing) or decreased saturation on current oxygen supplement, check chest film and arterial blood gas chest x‐ray/ arterial blood gas and call MD for possible bilevel positive airway pressure and repeating the echo. Intensive care unit (ICU) transfer as needed. According to his sister, his resuscitation preference was full code.

Given the new protocol instituted since the deployment of the EWS, the MSW reviewed the chart and found that the patient's sister, who lived locally and was the emergency contact, had been incorrectly identified as the surrogate. In a prior hospitalization, Mr. Smith had named his brother as his surrogate, as the patient felt strongly that his sister would not make good decisions for him. The following day, the SCT met with Mr. Smith, who articulated his desire to change his care directive to DNR. He also asked for a full palliative consult when his brother could come in (3 days later). During the consult, his brother learned, for the first time, exactly what heart failure was, and what to anticipate over the next months and years. The 2 brothers completed an advance directive granting Mr. Smith's brother a durable power of attorney including a request for a palliative approach to end‐stage illness. They also completed a physician order for life sustaining treatment, for DNR and limited intervention. Mr. Smith stated, When I go, I'm gone, and recalled that his mother and uncle had protracted illnesses, adding that I don't want to stay alive if I'm disabled like that.

In this example, the SCT was able to identify the correct surrogate and clarify the patient's resuscitation preference. Without SCT, if this patient had deteriorated unexpectedly, the sister would have insisted on treatment that was inconsistent with Mr. Smith's wishes. The interventions as a result of the alert also led the patient and his brother to begin discussing the medical goals of treatment openly and reach understanding about the patient's chronic and progressive conditions.

CASE STUDY 2: TRANSITION TO HOME‐BASED HOSPICE

Mr. North was a 71‐year‐old man admitted for sepsis due to pneumonia. He had a history of temporal arteritis, systemic lupus erythematosus, prostate cancer, squamous cell lung cancer, and chronic leg ulcers. Delirious at the time of admission, he triggered an alert at 6 _am, shortly after admission to the ward. He was hypotensive and was transferred to the ICU.

The SCT reviewed the case and judged that he met criteria for consultation. His wife readily agreed to meet to discuss goals and plan of care. She had been taking care of him at home, and was overwhelmed by his physical needs as well as his worsening memory loss and agitation. She had not been able to bring him to the clinic for almost 2 years, and he had refused entry to the home health nurse. During the palliative consult, Mr. North was lucid enough to state his preference for comfort‐focused care, and his desire not to return to the hospital. Mrs. North accepted a plan for home hospice, with increased attendant care at home.

This case illustrates the benefit of the EWS in identifying patients whose chronic condition has progressed, and who would benefit from a palliative consult to clarify goals of care. Practice variation, the complexity of multiple medical problems, and the urgency of the acute presentation may obscure or delay the need for clarifying goals of care. A structured approach provided by the EWS workflow, as it did in this case, helps to ensure that these discussions are occurring with the appropriate patients at the appropriate times.

CASE STUDY 3: RESOLVING MD‐TO‐MD MISCOMMUNICATION

Mr. Joseph was an 89‐year‐old male hospitalized for a hip fracture. He had a history of atrial fibrillation, prostate cancer with bone metastases, radiation‐induced lung fibrosis, stroke, and advanced dementia. His initial admission order was DNR, but this was changed after surgery to full code and remained so. The next few days were relatively uneventful until the alert triggered. By then, the hospitalist attending him had changed 3 times. The social worker reviewed Mr. Joseph's records and determined that a palliative consult had taken place previously at another Kaiser Permanente facility, and that the prior code status was DNR. Although Mr. Joseph's admission care directive was DNR, this was switched to full code for surgery. However, the care directive was not changed back, nor was a discussion held to discuss his preference in case of a complication related to surgery. Meanwhile, he was having increasing respiratory problems due to aspiration and required noninvasive ventilation.

Consequently, the SCT reviewed the alerts from the previous 24 hours and determined that further investigation and discussion were required. When the hospitalist was called, the SCT discovered that the hospitalist had assumed the change to full code had been made by 1 of the previous attending physicians; he also informed the SCT that Mr. Joseph would likely need intubation. The SCT decided to go see the patient and, on approaching the room, saw Mr. Joseph's son waiting outside. The son was asked how things were going, and replied, We all knew that 1 day he would deteriorate, we just want to make sure he is comfortable. Clearly, the full code status did not reflect the Mr. Joseph's wishes, so this was clarified and the hospitalist was called immediately to change the care directive. The SCT met with the man's son and wife, educating them about aspiration and what to expect. They definitely wished a gentle approach for Mr. Joseph, and it was decided to continue current care, without escalation, until the morning. This was to allow the other son to be informed of his father's condition and to see if his status would improve. The next morning the SCT met with the family at the room, and the patient was placed on comfort measures.

This case illustrates 3 points. One, Mr. Joseph's status was changed to full code during surgery without addressing his preferences should he develop a complication during the postoperative period. Two, when the hospitalist saw the full code order in the electronic record, it was assumed someone else had had a discussion with the patient and his family. Lastly, although a social worker performed a chart review, the full picture only emerged after the entire SCT became involved. Therefore, even in the presence of an EWS with associated protocols, important details can be missed, highlighting the need to build redundancy into workflows.

CASE STUDY 4: RELUCTANCE TO INVOLVE PALLIATIVE CARE TEAM

Mrs. Wood, a bed‐bound 63‐year‐old with end‐stage heart failure, was admitted to the hospital with respiratory failure. She had met with a life care planning facilitator as well as a palliative physician previously but refused to discuss end‐of‐life options. She felt she would always do well and her husband felt the same way. During this admission a routine palliative referral was made, but she and her husband refused. The chaplain visited often and then the patient took a turn for the worse, triggering an alert and was transferred to the ICU.

The hospitalist did not feel a SCT consult was indicated based on prior discussions. However, the SCT reviewed the records and felt an intervention was needed. The patient, now obtunded, had worsening renal failure and required continuous pressor infusions. The chaplain spoke with Mr. Wood, who felt a consult was appropriate. Mrs. Wood was no longer able to make decisions, and her husband needed more information about what to expect. At the end of the discussion, he decided on comfort care, and his wife expired peacefully in the hospital.

This case illustrates that, although initially the primary attending may feel a palliative consult is not helpful and possibly detrimental to the patient's care under usual circumstances, decisions may change as the patient's condition changes. The EWS alert helped the SCT recognize the drastic change in the patient's condition and the need to support the patient's family. The family had been resistant, but the SCT was able to help the family transition to a palliative approach with its gentle contact and being clear about its role to provide support regardless of their decision.

CASE STUDY 5: ALERT FACILITATES TRANSITION TO OUTPATIENT PALLIATIVE CARE

Mr. Jones was an 82‐year‐old gentleman who had a recent episode of gastrointestinal bleeding while on vacation. He was transferred by air ambulance to the hospital and developed delirium and agitation. His evaluation revealed that he had polycythemia vera and a recently made diagnosis of mild dementia.

In this case, the SCT reviewed the chart not because of an alert, but because the hospitalist noted that Mr. Jones had a very high severity of illness score on admission. When the SCT arrived at Mr. Jones's room, 3 family members were present. His wife appeared to be very frail and was too emotional to make decisions. The children present at the bedside were new to the problems at hand but wanted to help. The SCT team educated the family about his current disease state, the general disease trajectory, and what to expect. They explored the patient's values and any indicators of what his care preference would be if he could communicate it. The SCT established a life care plan at that visit. Based upon Mr. Jones's own wishes and values, he was made DNR with limited interventions. He survived the hospitalization and was followed by the outpatient palliative care clinic as well as by hematology.

This case illustrates 2 facets: a high severity of illness score led to consultation even without an alert. Following this, the SCT could take on a taskarriving at a life care plan by exploring valuesthat is difficult and time consuming for a busy hospitalist. It also illustrates that patients may elect to obtain other options, in this case, outpatient palliative care.

FUTURE DIRECTIONS

Our team has also started a quantitative evaluation process. The major limitation we face in this effort is that, unlike physiologic or health services measures (eg, tachycardia, hospital length of stay, mortality), the key measures for assessing the quality of palliative and end‐of‐life care need to be extracted by manual chart review. Our approach is based on the palliative and end‐of‐life care measures endorsed by the National Quality Forum,[23] which are described in greater detail in the appendix. As is the case with other outcomes, and as described in the article by Escobar et al.,[21] we will be employing a difference‐in‐differences approach as well as multivariate matching[24, 25, 26] to evaluate effectiveness of the intervention. Because of the high costs of manual chart review, we will be reviewing randomly selected charts of patients who triggered an alert at the 2 pilot sites as well as matched comparison patient charts at the remaining 19 KPNC hospitals. Table 1 provides preliminary data we gathered to pilot the brief chart review instrument that will be used for evaluating changes in supportive care in the regional rollout. Data are from a randomly selected cohort of 150 patients who reached the alert threshold at the 2 pilot sites between November 13, 2013 and June 30, 2014. After removing 3 records with substantial missing data, we were able to find 146 matched patients at the remaining 19 KPNC hospitals during the same time period. Matched patients were selected from those patients who had a virtual alert based on retrospective data. Table 1 shows that, compared to the other KPNC hospitals, the quality of these 6 aspects of supportive care was better at the pilot sites.

Table 1.Matched Analyses of Six Supportive Care Quality Measures
Hospital*	1	2	1+2 combined	Remaining 19	P (1)	P(2)	P(1+2)
NOTE: See text for additional details. The patients at the remaining 19 hospitals were identified based on their retrospective (virtual) deterioration probabilities and then matched to the patients at the pilot sites. The matching algorithm specified exact matches for these variables: alert threshold reached or not; sex; Kaiser Permanente membership status; had the patient been in the intensive care unit prior to the first alert; and care directive prior to the alert (full code vs not full code). Once potential matches were found using the above, the algorithm found the closest match for the following variables: deterioration probability, age, comorbidity burden, and admission illness severity. Statistical comparisons are as follows: P(1): P value for comparison of pilot hospital 1 versus remaining 19 Kaiser Permanente Northern California hospitals; P(2), as per P(1), but for pilot hospital 2; P(1+2), both pilot hospitals' data combined. For continuous variables, numbers shown are mean standard deviation. Numbers in bold italics* are those that were significantly different. Deterioration risk is generated by the early warning system. It is the probability that a patient will require transfer to the intensive care unit within the next 12 hours. Interventions are initiated when this risk is 8%. LAPS2 = admission Laboratory‐based Acute Physiology Score, version 2; measure of acute instability where the higher the score, the greater the degree of physiologic derangement. Patients with LAPS2 110 are very unstable. See citation 20 for additional details. COPS2 = Comorbidity Point Score, version 2; measure of chronic disease burden over preceding 12 months that is assigned to all Kaiser Permanente Northern California members on a monthly basis. The higher the score, the greater the chronic illness burden. Patients with COPS2 65 have a significant comorbid illness burden. See citation 20 for additional details. ‖Refers to 30 day mortality. Indicates whether documentation preceding an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. #Indicates whether documentation immediately following an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. **Refers to whether chart documentation indicated that the patient's family or agent were updated about the patient's condition within 24 hours after an alert. Refers to whether chart documentation indicated that a discussion occurred regarding the patient's goals of care occurred within 24 hours after an alert. Indicates whether a palliative care consultation occurred within 24 hours after an alert.
N	73	74	147	146
Age (y)	69.3 14.4	66.4 15.3	67.8 14.8	67.4 14.7	0.37	0.62	0.82
Male (%)	39 (53.4%)	43 (58.1%)	82 (55.8%)	82 (56.2%)	0.70	0.78	0.95
Deterioration risk (%)	20.0 14.3	17.4 11.6	18.7 13.0	18.8 13.6	0.54	0.44	0.94
LAPS2	113 38	102 39	107 39	107 38	0.28	0.38	0.9
COPS2	69 52	66 52	67 52	66 51	0.75	1.00	0.85
Died (%)‖	17 (23.3%)	15 (20.3%)	32 (21.8%)	24 (16.4%)	0.22	0.48	0.25
Agent identified prior	*28 (38.4%)*	18 (24.3%)	*46 (31.3%)*	21 (14.4%)	*<0.001*	0.07	*0.001*
Agent identified after#	*46 (63.0%)*	*39 (52.7%)*	*85 (57.8%)*	*28 (19.4%)*	*<0.001*	*<0.001*	*<0.001*
Updating within 24 hours**	32 (43.8%)	*45 (60.8%)*	*77 (52.4%)*	*59 (40.4%)*	0.63	0.00	0.04
Goals of care discussion	20 (27.4%)	*37 (50.0%)*	*57 (38.8%)*	*32 (21.9%)*	0.37	*0.001*	*0.002*
Palliative care consult	19 (26.0%)	49 (66.2%)	*68 (46.3%)*	*35 (24.0%)*	0.74	*<0.001*	*<0.001*
Spiritual support offered	27 (37.0%)	30 (40.5%)	57 (38.8%)	43 (29.4%)	0.26	0.10	0.09

CONCLUSION

Although we continue to review our care processes, we feel that our overall effort has been successful. Nonetheless, it is important to consider a number of limitations to the generalizability of our approach. First, our work has taken place in the context of a highly integrated care delivery system where both information transfer as well as referral from the inpatient to the outpatient setting can occur easily. Second, because the pilot sites were among the first KPNC hospitals to begin implementing the Respecting Choices model, they undoubtedly had less ground to cover than hospitals beginning with less infrastructure. Third, because of resource limitations, our ability to capture process data is limited. Lastly, both sites were able to obtain resources to expand necessary coverage, which might not be possible in many settings.

In conclusion, we made a conscious decision to incorporate palliative care into the planning for the deployment of the alert system. Further, we made this decision explicit, informing all caregivers that providing palliative care that adheres to the Respecting Choices model would be essential. We have found that integration of the SCT, the EWS, and routine hospital operations can be achieved. Clinician and patient acceptance of the Respecting Choices component has been excellent. We consider 3 elements to be critical for this process, and these elements form an integral component of the expansion of the early warning system to the remaining 19 KPNC hospitals. The first is careful planning, which includes instructing RRT first responders on their role in the process of ensuring the respect of patient preferences. Second, having social workers available 24 hours a day/7 days a week as backup for busy hospitalists, is essential. Finally, as is described by Dummett et al.,[27] including reminders regarding patient preferences in the documentation process (by embedding it in an automated note template) is also very important.

Acknowledgements

The authors thank Dr. Michelle Caughey, Dr. Philip Madvig, Dr. Patricia Conolly, Ms. Barbara Crawford, and Ms. Melissa Stern for their administrative support, Dr. Tracy Lieu for reviewing the manuscript, and Ms. Rachel Lesser for formatting the manuscript.

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Foundation played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. None of the other sponsors had any involvement in our decision to submit this manuscript or in the determination of its contents. None of the authors have any conflicts of interest to declare of relevance to this work.

APPENDIX 1 Key measures to assess the quality of supportive care extracted by manual chart review

Chart review question	Outcome desired	Outcome measured	Rationale for selecting this outcome
Was the patient's decision‐maker documented following the alert? If yes: Time/date of documentation.	Timely identification and documentation of the patient's decision‐maker immediately following the alert	Whether the patient's decision‐maker was clearly identified and documented by a member of the treatment team (nurse, physician, and/or rapid response team) following the alert. This outcome is measured independently of whether the patient's decision‐maker was already documented prior to the alert.	Clear documentation facilitates the notification of a patient's family/decision‐maker in a timely manner to enhance communication and clinical decision‐making to make sure that the patient's wishes and preferences are honored.
Was the patient's decision‐maker/family notified or was there an attempt to notify the patient's decision‐maker regarding the changes in the patient's condition following the alert? If yes: Time/date of notification/attempted contact.	Providing patient's family members/decision‐maker with an update in the patient's clinical condition following the alert	Whether the medical team notified or attempted to contact the patient's family/decision‐maker to provide an update in the patient's clinical condition following the alert.	Providing timely updates when a patient's clinical status changes enhances communication and helps to proactively involve patients and families in the decision‐making process.
Was there a goals of care discussion following the alert? If yes: Time/date of discussion	To clarify and to honor individual patient's goals of care	Whether a goals of care discussion was initiated after the alert was issued. Criteria for Goals of Care discussion included any/all of the following: Specific language in the documentation that stated verbatim Goals of Care Discussion Providing prognosis and treatment options; eliciting preferences; AND documenting decisions made and preferences as a result of the discussion.	Goals of care discussions actively involve patients and families in the decision‐making process to ensure that their wishes and preferences are clearly documented and followed.
Was there a palliative care consultation during the patient's hospitalization?	To provide comprehensive supportive care to patients and their families/loved ones.	Whether palliative care was consulted during the patient's hospitalization	The palliative care team plays an important role in helping patients/families make decisions, providing support, and ensuring that patients symptoms are addressed and properly managed
Was spiritual support offered to the patient and/or their family/loved during the patient's hospitalization?	To offer and to provide spiritual support to patients and their families/loved ones	Whether the patient/family was offered spiritual support during the patient's hospitalization	Spiritual support has been recognized as an important aspect of quality EOL care

APPENDIX 2 Respecting Choices, A Staged Approach to Advance Care Planning

Respecting Choices is a staged approach to advance care planning, where conversations begin when people are healthy and continue to occur throughout life.

Our Life Care Planning service consists ofthree distinct steps.

My Values: First Steps is appropriate for all adults, but should definitely be initiated as a component of routine healthcare for those over the age of 55. The goals of First Steps are to motivate individuals to learn more about the importance of Life Care Planning, select a healthcare decision maker, and complete a basic written advance directive.
My Choices: Next Steps is for patients with chronic, progressive illness who have begun to experience a decline in functional status or frequent hospitalizations. The goals of this stage of planning are to assist patients in understanding a) the progression of their illness, b) potential complications, and c) specific life‐sustaining treatments that may be required if their illness progresses. Understanding life‐sustaining treatments includes each treatment's benefits, burdens, and alternatives. With this understanding members will be better able to express what situations (e.g. complications or bad outcomes) would cause them to want to change their plan of care.Additionally, the individual's healthcare agent(s) and other loved ones are involved in the planning process so that they can be prepared to make decisions, if necessary, and to support the plan of care developed.
My Care: Advanced Steps is intended for frail elders or others whose death in the next 12 months would not be surprising. It helps patients and their agent make specific and timely life‐sustaining treatment decisions that can be converted to medical orders to guide the actions of healthcare providers and be consistent with the goals of the individual.

(Reference: http://www.gundersenhealth.org/respecting-choices).

APPENDIX 3 Pilot site Palliative Care Referral Criteria

Automatic palliative care consults for adults at Sacramento site are as follows:

30 day readmits or >3 ED or acute readmissions in past year for CHF or COPD that have no Advance Directive and are not followed by Chronic Care Management
Aspiration
CVA with poor prognosis for regaining independence
Hip fracture patients not weight bearing on post‐operative day 2
Code blue survivor
Skilled Nursing Facility resident with sepsis and or dementia
Active hospice patients
Sepsis patients with 10 or more ICD codes in the problem list

Potential palliative care consults for adults at Sacramento pilot site are as follows:

Morbid obesity complicated by organ damage (e.g., congestive heart failure, refractory liver disease, chronic renal disease)
Severe chronic kidney disease and/or congestive heart failure with poor functional status (chair or bed bound)
Patient with pre‐operative arteriovenous fistulas and poor functional status, congestive heart failure, or age>80
End stage liver disease with declining functional status, poor odds of transplant

Files

References

Institute of Medicine of the National Academies. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: Institute of Medicine of the National Academies; 2014.
Partners LR. Final chapter: Californians' attitudes and experiences with death and dying. California HealthCare Foundation website. Available at: http://www.chcf.org/publications/2012/02/final‐chapter‐death‐dying. Published February 2012. Accessed July 14, 2015.
Rozenbaum EA, Shenkman L. Predicting outcome of inhospital cardiopulmonary resuscitation. Crit Care Med. 1988;16(6):583–586.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Jones DA, McIntyre T, Baldwin I, Mercer I, Kattula A, Bellomo R. The medical emergency team and end‐of‐life care: a pilot study. Crit Care Resusc. 2007;9(2):151–156.
Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S. The Medical Emergency Team System and not‐for‐resuscitation orders: results from the MERIT study. Resuscitation. 2008;79(3):391–397.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Knott CI, Psirides AJ, Young PJ, Sim D. A retrospective cohort study of the effect of medical emergency teams on documentation of advance care directives. Crit Care Resusc. 2011;13(3):167–174.
Coventry C, Flabouris A, Sundararajan K, Cramey T. Rapid response team calls to patients with a pre‐existing not for resuscitation order. Resuscitation. 2013;84(8):1035–1039.
Downar J, Barua R, Rodin D, et al. Changes in end of life care 5 years after the introduction of a rapid response team: a multicentre retrospective study. Resuscitation. 2013;84(10):1339–1344.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Sundararajan K, Flabouris A, Keeshan A, Cramey T. Documentation of limitation of medical therapy at the time of a rapid response team call. Aust Health Rev. 2014;38(2):218–222.
Visser P, Dwyer A, Moran J, et al. Medical emergency response in a sub‐acute hospital: improving the model of care for deteriorating patients. Aust Health Rev. 2014;38(2):169–176.
Respecting Choices advance care planning. Available at: http://www.gundersenhealth.org/respecting‐choices. Gundersen Health System website. Accessed March 28, 2015.
Escobar G, Dellinger RP. Early detection, prevention, and mitigation of critical illness outside intensive care settings. J Hosp Med. 2016;11:000–000.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Department of Health and Human Services. Palliative care and end‐of‐life care—a consensus report. National Quality Forum website. Available at: http://www.qualityforum.org/projects/palliative_care_and_end‐of‐life_care.aspx. Accessed April 1, 2015.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study: Eli Lilly working paper. Available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed January 24, 2013.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dummett BA, Adams C, Scruth E, Liu V, Guo M, Escobar G. Incorporating an early detection system into routine clinical practice in two community hospitals. J Hosp Med. 2016;11:000–000.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S40-S47

Read more about Critical Illness Outside the ICU

Sections

Author(s)

Zoe Sutton, RN, MSN, PNP

Author(s)

Zoe Sutton, RN, MSN, PNP

Files

Files

Article PDF

Article PDF

IMPLEMENTATION APPROACH

CASE STUDY 1: SURROGATE IDENTIFICATION

CASE STUDY 2: TRANSITION TO HOME‐BASED HOSPICE

CASE STUDY 3: RESOLVING MD‐TO‐MD MISCOMMUNICATION

CASE STUDY 4: RELUCTANCE TO INVOLVE PALLIATIVE CARE TEAM

CASE STUDY 5: ALERT FACILITATES TRANSITION TO OUTPATIENT PALLIATIVE CARE

FUTURE DIRECTIONS

Table 1.Matched Analyses of Six Supportive Care Quality Measures
Hospital*	1	2	1+2 combined	Remaining 19	P (1)	P(2)	P(1+2)
NOTE: See text for additional details. The patients at the remaining 19 hospitals were identified based on their retrospective (virtual) deterioration probabilities and then matched to the patients at the pilot sites. The matching algorithm specified exact matches for these variables: alert threshold reached or not; sex; Kaiser Permanente membership status; had the patient been in the intensive care unit prior to the first alert; and care directive prior to the alert (full code vs not full code). Once potential matches were found using the above, the algorithm found the closest match for the following variables: deterioration probability, age, comorbidity burden, and admission illness severity. Statistical comparisons are as follows: P(1): P value for comparison of pilot hospital 1 versus remaining 19 Kaiser Permanente Northern California hospitals; P(2), as per P(1), but for pilot hospital 2; P(1+2), both pilot hospitals' data combined. For continuous variables, numbers shown are mean standard deviation. Numbers in bold italics* are those that were significantly different. Deterioration risk is generated by the early warning system. It is the probability that a patient will require transfer to the intensive care unit within the next 12 hours. Interventions are initiated when this risk is 8%. LAPS2 = admission Laboratory‐based Acute Physiology Score, version 2; measure of acute instability where the higher the score, the greater the degree of physiologic derangement. Patients with LAPS2 110 are very unstable. See citation 20 for additional details. COPS2 = Comorbidity Point Score, version 2; measure of chronic disease burden over preceding 12 months that is assigned to all Kaiser Permanente Northern California members on a monthly basis. The higher the score, the greater the chronic illness burden. Patients with COPS2 65 have a significant comorbid illness burden. See citation 20 for additional details. ‖Refers to 30 day mortality. Indicates whether documentation preceding an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. #Indicates whether documentation immediately following an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. **Refers to whether chart documentation indicated that the patient's family or agent were updated about the patient's condition within 24 hours after an alert. Refers to whether chart documentation indicated that a discussion occurred regarding the patient's goals of care occurred within 24 hours after an alert. Indicates whether a palliative care consultation occurred within 24 hours after an alert.
N	73	74	147	146
Age (y)	69.3 14.4	66.4 15.3	67.8 14.8	67.4 14.7	0.37	0.62	0.82
Male (%)	39 (53.4%)	43 (58.1%)	82 (55.8%)	82 (56.2%)	0.70	0.78	0.95
Deterioration risk (%)	20.0 14.3	17.4 11.6	18.7 13.0	18.8 13.6	0.54	0.44	0.94
LAPS2	113 38	102 39	107 39	107 38	0.28	0.38	0.9
COPS2	69 52	66 52	67 52	66 51	0.75	1.00	0.85
Died (%)‖	17 (23.3%)	15 (20.3%)	32 (21.8%)	24 (16.4%)	0.22	0.48	0.25
Agent identified prior	*28 (38.4%)*	18 (24.3%)	*46 (31.3%)*	21 (14.4%)	*<0.001*	0.07	*0.001*
Agent identified after#	*46 (63.0%)*	*39 (52.7%)*	*85 (57.8%)*	*28 (19.4%)*	*<0.001*	*<0.001*	*<0.001*
Updating within 24 hours**	32 (43.8%)	*45 (60.8%)*	*77 (52.4%)*	*59 (40.4%)*	0.63	0.00	0.04
Goals of care discussion	20 (27.4%)	*37 (50.0%)*	*57 (38.8%)*	*32 (21.9%)*	0.37	*0.001*	*0.002*
Palliative care consult	19 (26.0%)	49 (66.2%)	*68 (46.3%)*	*35 (24.0%)*	0.74	*<0.001*	*<0.001*
Spiritual support offered	27 (37.0%)	30 (40.5%)	57 (38.8%)	43 (29.4%)	0.26	0.10	0.09

CONCLUSION

Acknowledgements

APPENDIX 1 Key measures to assess the quality of supportive care extracted by manual chart review

Chart review question	Outcome desired	Outcome measured	Rationale for selecting this outcome
Was the patient's decision‐maker documented following the alert? If yes: Time/date of documentation.	Timely identification and documentation of the patient's decision‐maker immediately following the alert	Whether the patient's decision‐maker was clearly identified and documented by a member of the treatment team (nurse, physician, and/or rapid response team) following the alert. This outcome is measured independently of whether the patient's decision‐maker was already documented prior to the alert.	Clear documentation facilitates the notification of a patient's family/decision‐maker in a timely manner to enhance communication and clinical decision‐making to make sure that the patient's wishes and preferences are honored.
Was the patient's decision‐maker/family notified or was there an attempt to notify the patient's decision‐maker regarding the changes in the patient's condition following the alert? If yes: Time/date of notification/attempted contact.	Providing patient's family members/decision‐maker with an update in the patient's clinical condition following the alert	Whether the medical team notified or attempted to contact the patient's family/decision‐maker to provide an update in the patient's clinical condition following the alert.	Providing timely updates when a patient's clinical status changes enhances communication and helps to proactively involve patients and families in the decision‐making process.
Was there a goals of care discussion following the alert? If yes: Time/date of discussion	To clarify and to honor individual patient's goals of care	Whether a goals of care discussion was initiated after the alert was issued. Criteria for Goals of Care discussion included any/all of the following: Specific language in the documentation that stated verbatim Goals of Care Discussion Providing prognosis and treatment options; eliciting preferences; AND documenting decisions made and preferences as a result of the discussion.	Goals of care discussions actively involve patients and families in the decision‐making process to ensure that their wishes and preferences are clearly documented and followed.
Was there a palliative care consultation during the patient's hospitalization?	To provide comprehensive supportive care to patients and their families/loved ones.	Whether palliative care was consulted during the patient's hospitalization	The palliative care team plays an important role in helping patients/families make decisions, providing support, and ensuring that patients symptoms are addressed and properly managed
Was spiritual support offered to the patient and/or their family/loved during the patient's hospitalization?	To offer and to provide spiritual support to patients and their families/loved ones	Whether the patient/family was offered spiritual support during the patient's hospitalization	Spiritual support has been recognized as an important aspect of quality EOL care

APPENDIX 2 Respecting Choices, A Staged Approach to Advance Care Planning

Respecting Choices is a staged approach to advance care planning, where conversations begin when people are healthy and continue to occur throughout life.

Our Life Care Planning service consists ofthree distinct steps.

My Values: First Steps is appropriate for all adults, but should definitely be initiated as a component of routine healthcare for those over the age of 55. The goals of First Steps are to motivate individuals to learn more about the importance of Life Care Planning, select a healthcare decision maker, and complete a basic written advance directive.
My Choices: Next Steps is for patients with chronic, progressive illness who have begun to experience a decline in functional status or frequent hospitalizations. The goals of this stage of planning are to assist patients in understanding a) the progression of their illness, b) potential complications, and c) specific life‐sustaining treatments that may be required if their illness progresses. Understanding life‐sustaining treatments includes each treatment's benefits, burdens, and alternatives. With this understanding members will be better able to express what situations (e.g. complications or bad outcomes) would cause them to want to change their plan of care.Additionally, the individual's healthcare agent(s) and other loved ones are involved in the planning process so that they can be prepared to make decisions, if necessary, and to support the plan of care developed.
My Care: Advanced Steps is intended for frail elders or others whose death in the next 12 months would not be surprising. It helps patients and their agent make specific and timely life‐sustaining treatment decisions that can be converted to medical orders to guide the actions of healthcare providers and be consistent with the goals of the individual.

(Reference: http://www.gundersenhealth.org/respecting-choices).

APPENDIX 3 Pilot site Palliative Care Referral Criteria

Automatic palliative care consults for adults at Sacramento site are as follows:

30 day readmits or >3 ED or acute readmissions in past year for CHF or COPD that have no Advance Directive and are not followed by Chronic Care Management
Aspiration
CVA with poor prognosis for regaining independence
Hip fracture patients not weight bearing on post‐operative day 2
Code blue survivor
Skilled Nursing Facility resident with sepsis and or dementia
Active hospice patients
Sepsis patients with 10 or more ICD codes in the problem list

Potential palliative care consults for adults at Sacramento pilot site are as follows:

Morbid obesity complicated by organ damage (e.g., congestive heart failure, refractory liver disease, chronic renal disease)
Severe chronic kidney disease and/or congestive heart failure with poor functional status (chair or bed bound)
Patient with pre‐operative arteriovenous fistulas and poor functional status, congestive heart failure, or age>80
End stage liver disease with declining functional status, poor odds of transplant

IMPLEMENTATION APPROACH

CASE STUDY 1: SURROGATE IDENTIFICATION

CASE STUDY 2: TRANSITION TO HOME‐BASED HOSPICE

CASE STUDY 3: RESOLVING MD‐TO‐MD MISCOMMUNICATION

CASE STUDY 4: RELUCTANCE TO INVOLVE PALLIATIVE CARE TEAM

CASE STUDY 5: ALERT FACILITATES TRANSITION TO OUTPATIENT PALLIATIVE CARE

FUTURE DIRECTIONS

Table 1.Matched Analyses of Six Supportive Care Quality Measures
Hospital*	1	2	1+2 combined	Remaining 19	P (1)	P(2)	P(1+2)
NOTE: See text for additional details. The patients at the remaining 19 hospitals were identified based on their retrospective (virtual) deterioration probabilities and then matched to the patients at the pilot sites. The matching algorithm specified exact matches for these variables: alert threshold reached or not; sex; Kaiser Permanente membership status; had the patient been in the intensive care unit prior to the first alert; and care directive prior to the alert (full code vs not full code). Once potential matches were found using the above, the algorithm found the closest match for the following variables: deterioration probability, age, comorbidity burden, and admission illness severity. Statistical comparisons are as follows: P(1): P value for comparison of pilot hospital 1 versus remaining 19 Kaiser Permanente Northern California hospitals; P(2), as per P(1), but for pilot hospital 2; P(1+2), both pilot hospitals' data combined. For continuous variables, numbers shown are mean standard deviation. Numbers in bold italics* are those that were significantly different. Deterioration risk is generated by the early warning system. It is the probability that a patient will require transfer to the intensive care unit within the next 12 hours. Interventions are initiated when this risk is 8%. LAPS2 = admission Laboratory‐based Acute Physiology Score, version 2; measure of acute instability where the higher the score, the greater the degree of physiologic derangement. Patients with LAPS2 110 are very unstable. See citation 20 for additional details. COPS2 = Comorbidity Point Score, version 2; measure of chronic disease burden over preceding 12 months that is assigned to all Kaiser Permanente Northern California members on a monthly basis. The higher the score, the greater the chronic illness burden. Patients with COPS2 65 have a significant comorbid illness burden. See citation 20 for additional details. ‖Refers to 30 day mortality. Indicates whether documentation preceding an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. #Indicates whether documentation immediately following an alert clearly specified who the patient's agent (decision‐maker or surrogate) was. **Refers to whether chart documentation indicated that the patient's family or agent were updated about the patient's condition within 24 hours after an alert. Refers to whether chart documentation indicated that a discussion occurred regarding the patient's goals of care occurred within 24 hours after an alert. Indicates whether a palliative care consultation occurred within 24 hours after an alert.
N	73	74	147	146
Age (y)	69.3 14.4	66.4 15.3	67.8 14.8	67.4 14.7	0.37	0.62	0.82
Male (%)	39 (53.4%)	43 (58.1%)	82 (55.8%)	82 (56.2%)	0.70	0.78	0.95
Deterioration risk (%)	20.0 14.3	17.4 11.6	18.7 13.0	18.8 13.6	0.54	0.44	0.94
LAPS2	113 38	102 39	107 39	107 38	0.28	0.38	0.9
COPS2	69 52	66 52	67 52	66 51	0.75	1.00	0.85
Died (%)‖	17 (23.3%)	15 (20.3%)	32 (21.8%)	24 (16.4%)	0.22	0.48	0.25
Agent identified prior	*28 (38.4%)*	18 (24.3%)	*46 (31.3%)*	21 (14.4%)	*<0.001*	0.07	*0.001*
Agent identified after#	*46 (63.0%)*	*39 (52.7%)*	*85 (57.8%)*	*28 (19.4%)*	*<0.001*	*<0.001*	*<0.001*
Updating within 24 hours**	32 (43.8%)	*45 (60.8%)*	*77 (52.4%)*	*59 (40.4%)*	0.63	0.00	0.04
Goals of care discussion	20 (27.4%)	*37 (50.0%)*	*57 (38.8%)*	*32 (21.9%)*	0.37	*0.001*	*0.002*
Palliative care consult	19 (26.0%)	49 (66.2%)	*68 (46.3%)*	*35 (24.0%)*	0.74	*<0.001*	*<0.001*
Spiritual support offered	27 (37.0%)	30 (40.5%)	57 (38.8%)	43 (29.4%)	0.26	0.10	0.09

CONCLUSION

Acknowledgements

APPENDIX 1 Key measures to assess the quality of supportive care extracted by manual chart review

Chart review question	Outcome desired	Outcome measured	Rationale for selecting this outcome
Was the patient's decision‐maker documented following the alert? If yes: Time/date of documentation.	Timely identification and documentation of the patient's decision‐maker immediately following the alert	Whether the patient's decision‐maker was clearly identified and documented by a member of the treatment team (nurse, physician, and/or rapid response team) following the alert. This outcome is measured independently of whether the patient's decision‐maker was already documented prior to the alert.	Clear documentation facilitates the notification of a patient's family/decision‐maker in a timely manner to enhance communication and clinical decision‐making to make sure that the patient's wishes and preferences are honored.
Was the patient's decision‐maker/family notified or was there an attempt to notify the patient's decision‐maker regarding the changes in the patient's condition following the alert? If yes: Time/date of notification/attempted contact.	Providing patient's family members/decision‐maker with an update in the patient's clinical condition following the alert	Whether the medical team notified or attempted to contact the patient's family/decision‐maker to provide an update in the patient's clinical condition following the alert.	Providing timely updates when a patient's clinical status changes enhances communication and helps to proactively involve patients and families in the decision‐making process.
Was there a goals of care discussion following the alert? If yes: Time/date of discussion	To clarify and to honor individual patient's goals of care	Whether a goals of care discussion was initiated after the alert was issued. Criteria for Goals of Care discussion included any/all of the following: Specific language in the documentation that stated verbatim Goals of Care Discussion Providing prognosis and treatment options; eliciting preferences; AND documenting decisions made and preferences as a result of the discussion.	Goals of care discussions actively involve patients and families in the decision‐making process to ensure that their wishes and preferences are clearly documented and followed.
Was there a palliative care consultation during the patient's hospitalization?	To provide comprehensive supportive care to patients and their families/loved ones.	Whether palliative care was consulted during the patient's hospitalization	The palliative care team plays an important role in helping patients/families make decisions, providing support, and ensuring that patients symptoms are addressed and properly managed
Was spiritual support offered to the patient and/or their family/loved during the patient's hospitalization?	To offer and to provide spiritual support to patients and their families/loved ones	Whether the patient/family was offered spiritual support during the patient's hospitalization	Spiritual support has been recognized as an important aspect of quality EOL care

APPENDIX 2 Respecting Choices, A Staged Approach to Advance Care Planning

Respecting Choices is a staged approach to advance care planning, where conversations begin when people are healthy and continue to occur throughout life.

Our Life Care Planning service consists ofthree distinct steps.

My Values: First Steps is appropriate for all adults, but should definitely be initiated as a component of routine healthcare for those over the age of 55. The goals of First Steps are to motivate individuals to learn more about the importance of Life Care Planning, select a healthcare decision maker, and complete a basic written advance directive.
My Choices: Next Steps is for patients with chronic, progressive illness who have begun to experience a decline in functional status or frequent hospitalizations. The goals of this stage of planning are to assist patients in understanding a) the progression of their illness, b) potential complications, and c) specific life‐sustaining treatments that may be required if their illness progresses. Understanding life‐sustaining treatments includes each treatment's benefits, burdens, and alternatives. With this understanding members will be better able to express what situations (e.g. complications or bad outcomes) would cause them to want to change their plan of care.Additionally, the individual's healthcare agent(s) and other loved ones are involved in the planning process so that they can be prepared to make decisions, if necessary, and to support the plan of care developed.
My Care: Advanced Steps is intended for frail elders or others whose death in the next 12 months would not be surprising. It helps patients and their agent make specific and timely life‐sustaining treatment decisions that can be converted to medical orders to guide the actions of healthcare providers and be consistent with the goals of the individual.

(Reference: http://www.gundersenhealth.org/respecting-choices).

APPENDIX 3 Pilot site Palliative Care Referral Criteria

Automatic palliative care consults for adults at Sacramento site are as follows:

30 day readmits or >3 ED or acute readmissions in past year for CHF or COPD that have no Advance Directive and are not followed by Chronic Care Management
Aspiration
CVA with poor prognosis for regaining independence
Hip fracture patients not weight bearing on post‐operative day 2
Code blue survivor
Skilled Nursing Facility resident with sepsis and or dementia
Active hospice patients
Sepsis patients with 10 or more ICD codes in the problem list

Potential palliative care consults for adults at Sacramento pilot site are as follows:

Morbid obesity complicated by organ damage (e.g., congestive heart failure, refractory liver disease, chronic renal disease)
Severe chronic kidney disease and/or congestive heart failure with poor functional status (chair or bed bound)
Patient with pre‐operative arteriovenous fistulas and poor functional status, congestive heart failure, or age>80
End stage liver disease with declining functional status, poor odds of transplant

References

Institute of Medicine of the National Academies. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: Institute of Medicine of the National Academies; 2014.
Partners LR. Final chapter: Californians' attitudes and experiences with death and dying. California HealthCare Foundation website. Available at: http://www.chcf.org/publications/2012/02/final‐chapter‐death‐dying. Published February 2012. Accessed July 14, 2015.
Rozenbaum EA, Shenkman L. Predicting outcome of inhospital cardiopulmonary resuscitation. Crit Care Med. 1988;16(6):583–586.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Jones DA, McIntyre T, Baldwin I, Mercer I, Kattula A, Bellomo R. The medical emergency team and end‐of‐life care: a pilot study. Crit Care Resusc. 2007;9(2):151–156.
Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S. The Medical Emergency Team System and not‐for‐resuscitation orders: results from the MERIT study. Resuscitation. 2008;79(3):391–397.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Knott CI, Psirides AJ, Young PJ, Sim D. A retrospective cohort study of the effect of medical emergency teams on documentation of advance care directives. Crit Care Resusc. 2011;13(3):167–174.
Coventry C, Flabouris A, Sundararajan K, Cramey T. Rapid response team calls to patients with a pre‐existing not for resuscitation order. Resuscitation. 2013;84(8):1035–1039.
Downar J, Barua R, Rodin D, et al. Changes in end of life care 5 years after the introduction of a rapid response team: a multicentre retrospective study. Resuscitation. 2013;84(10):1339–1344.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Sundararajan K, Flabouris A, Keeshan A, Cramey T. Documentation of limitation of medical therapy at the time of a rapid response team call. Aust Health Rev. 2014;38(2):218–222.
Visser P, Dwyer A, Moran J, et al. Medical emergency response in a sub‐acute hospital: improving the model of care for deteriorating patients. Aust Health Rev. 2014;38(2):169–176.
Respecting Choices advance care planning. Available at: http://www.gundersenhealth.org/respecting‐choices. Gundersen Health System website. Accessed March 28, 2015.
Escobar G, Dellinger RP. Early detection, prevention, and mitigation of critical illness outside intensive care settings. J Hosp Med. 2016;11:000–000.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Department of Health and Human Services. Palliative care and end‐of‐life care—a consensus report. National Quality Forum website. Available at: http://www.qualityforum.org/projects/palliative_care_and_end‐of‐life_care.aspx. Accessed April 1, 2015.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study: Eli Lilly working paper. Available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed January 24, 2013.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dummett BA, Adams C, Scruth E, Liu V, Guo M, Escobar G. Incorporating an early detection system into routine clinical practice in two community hospitals. J Hosp Med. 2016;11:000–000.

References

Institute of Medicine of the National Academies. Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life. Washington, DC: Institute of Medicine of the National Academies; 2014.
Partners LR. Final chapter: Californians' attitudes and experiences with death and dying. California HealthCare Foundation website. Available at: http://www.chcf.org/publications/2012/02/final‐chapter‐death‐dying. Published February 2012. Accessed July 14, 2015.
Rozenbaum EA, Shenkman L. Predicting outcome of inhospital cardiopulmonary resuscitation. Crit Care Med. 1988;16(6):583–586.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Jones DA, McIntyre T, Baldwin I, Mercer I, Kattula A, Bellomo R. The medical emergency team and end‐of‐life care: a pilot study. Crit Care Resusc. 2007;9(2):151–156.
Chen J, Flabouris A, Bellomo R, Hillman K, Finfer S. The Medical Emergency Team System and not‐for‐resuscitation orders: results from the MERIT study. Resuscitation. 2008;79(3):391–397.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Knott CI, Psirides AJ, Young PJ, Sim D. A retrospective cohort study of the effect of medical emergency teams on documentation of advance care directives. Crit Care Resusc. 2011;13(3):167–174.
Coventry C, Flabouris A, Sundararajan K, Cramey T. Rapid response team calls to patients with a pre‐existing not for resuscitation order. Resuscitation. 2013;84(8):1035–1039.
Downar J, Barua R, Rodin D, et al. Changes in end of life care 5 years after the introduction of a rapid response team: a multicentre retrospective study. Resuscitation. 2013;84(10):1339–1344.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Sundararajan K, Flabouris A, Keeshan A, Cramey T. Documentation of limitation of medical therapy at the time of a rapid response team call. Aust Health Rev. 2014;38(2):218–222.
Visser P, Dwyer A, Moran J, et al. Medical emergency response in a sub‐acute hospital: improving the model of care for deteriorating patients. Aust Health Rev. 2014;38(2):169–176.
Respecting Choices advance care planning. Available at: http://www.gundersenhealth.org/respecting‐choices. Gundersen Health System website. Accessed March 28, 2015.
Escobar G, Dellinger RP. Early detection, prevention, and mitigation of critical illness outside intensive care settings. J Hosp Med. 2016;11:000–000.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Department of Health and Human Services. Palliative care and end‐of‐life care—a consensus report. National Quality Forum website. Available at: http://www.qualityforum.org/projects/palliative_care_and_end‐of‐life_care.aspx. Accessed April 1, 2015.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study: Eli Lilly working paper. Available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed January 24, 2013.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dummett BA, Adams C, Scruth E, Liu V, Guo M, Escobar G. Incorporating an early detection system into routine clinical practice in two community hospitals. J Hosp Med. 2016;11:000–000.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S40-S47

Page Number

S40-S47

Publications

Publications

Article Type

Display Headline

Early detection of critical illness outside the intensive care unit: Clarifying treatment plans and honoring goals of care using a supportive care team

Display Headline

Early detection of critical illness outside the intensive care unit: Clarifying treatment plans and honoring goals of care using a supportive care team

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Russ Granich, MD, South San Francisco Kaiser Permanente, 1200 El Camino Real, South San Francisco, CA 94080; Telephone: 650‐827‐6361; Fax: 650‐827‐6356; E‐mail: russ.granich@kp.org

Content Gating

Gated (full article locked unless allowed per User)

Alternative CME

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

Media Files

Program for Early Detection of Sepsis

Article Type

Changed

Mon, 01/30/2017 - 11:15

Display Headline

Implementation of a multicenter performance improvement program for early detection and treatment of severe sepsis in general medical–surgical wards

Author(s)

Andrew Odden, MD, SFHM

Laura Evans, MD, MSc

Snehal Gandhi, MD

Sean Townsend, MD

Mitchell Levy, MD

Sepsis, the body's systemic response to infection leading to organ failure, can occur in patients throughout the hospital. However, patients initially diagnosed with sepsis on the wards experience the highest mortality for several reasons, including delayed recognition and treatment, particularly when localized infections progress to shock and organ failure. Consequently, hospitals have responded by having nurses screen patients for signs and symptoms of sepsis to identify cases earlier and improve outcomes. The intent of this article, which is based on our experience with a multihospital implementation effort, was to describe potential reasons for ward patients' poor prognosis. We provide a toolkit for how hospitals can implement a severe sepsis quality improvement (QI) program in general medicalsurgical wards.

In a previous study, we reported on our international effort, the Surviving Sepsis Campaign's (SSC) Phase III performance improvement (PI) program, targeting selected guideline recommendations (6‐ and 24‐hour bundles) in the emergency department (ED), the Intensive Care Unit (ICU), and wards in 165 volunteer hospitals in the United States, Europe, and South America.[1] The program was associated with increased bundle compliance and decreased mortality over time.[1, 2] The SSC's Phase III program, which focused on improvement efforts primarily in the ED and ICU, also exposed a need to address the high mortality in ward patients.[3] Patients admitted to the ICU directly from the ED with severe sepsis had a mortality rate of 26%, whereas those transferred to the ICU from the ward had significantly higher mortality (40.3%).[3]

Although the reasons for the higher mortality rate among ward patients have not been studied, several factors may play a role. First, the diagnosis of severe sepsis may be delayed in ward patients because physicians and nurses may not recognize the progression to sepsis and/or because hospitalized patients may not present with obvious systemic manifestations of sepsis as they do in the ED (Table 1).[4] Second, ward patients may have differences in the timing of their presentation and concurrent conditions confounding the diagnosis.[5] Third, treatment may be delayed once the diagnosis is made on the ward. The ICU and ED are designed to provide rapid high‐acuity care, whereas the wards have fewer systems and resources for rapid delivery of care needed for severe sepsis. Finally, some patients on the ward may develop sepsis from nosocomial infection, which can portend a worse prognosis.[6]

Table 1.Presentation of Severe Sepsis in the Emergency Department and the Ward
	Emergency Department Presentation	Ward Presentation
Patient‐familyreported symptoms	I just feel sick, family reports disorientation, not eating	Currently hospitalized, family often not present, diagnosis may not be clear, baseline mental status unknown, lack of appetite may be linked to dislike of hospital food.
Systemic manifestations	Triage observed 2 or more signs of infection or patient reports temperature while at home plus additional finding on assessment.	Signs of infection may appear 1 at a time, hours apart, and may appear to be mild changes to staff or missed entirely due to staff discontinuity.
Organ dysfunction	Present on admission; triage nurse assesses for organ dysfunction.	Develops over hours or days; may be subtle or acute.
Laboratory study process	Ordered and evaluated within 1 hour.	Not routinely completed daily, may be ordered after physician evaluation or during rounds. Results within 34 hours.

The SSC Phase III results led to the launch of a QI program, known as the SSC Phase IV Sepsis on the Wards Collaborative, funded by the Gordon and Betty Moore Foundation. This program, a partnership between the Society of Critical Care Medicine and the Society of Hospital Medicine (SHM), targeted ward patients and focused on early recognition through protocol‐driven regular nurse screening. The program applied the SSC 2012 guidelines with a primary focus on the 3‐hour bundle (Table 2).[7] The framework used for this program was the Institute for Healthcare Improvement's Plan‐Do‐Study‐Act (PDSA) model of improvement.[8, 9] The collaborative design included learning sessions designed to motivate and support improvement.[10] The program began with 60 academic and community hospitals in 4 US regions. Participating sites were required to have prior hospital experience in sepsis performance improvement as well as a formal commitment of support from their EDs and ICUs.

Table 2.Surviving Sepsis Campaign 3‐Hour Severe Sepsis Bundle
To be completed within 3 hours of time of presentation
1. Measure lactate level
2. Obtain blood cultures prior to administration of antibiotics
3. Administer broad‐spectrum antibiotics
4. Administer 30 mL/kg crystalloid for hypotension or lactate 4 mmol/L (36 mg/dL)

We provided sites with a basic screening tool and guidance for routine severe sepsis screening, monitoring, and feedback (Figure 1). Because of the anticipated challenges of implementing routine nurse screening on every shift in all inpatient wards, participants identified 1 ward to pilot the every‐shift screening program. Each pilot ward refined the nurse screening process and developed site‐specific tools based on electronic health record (EHR) capability, informatics support, and available resources. After this initial phase, the program could be implemented in a hospital's remaining wards. The slogan adopted for the program was Screen every patient, every shift, every day.

Evaluation for severe sepsis screening tool. This checklist is designed to prompt the nurse to screen every patient during every shift for new signs of sepsis and organ dysfunction (Checklist is available at: http://www.survivingsepsis.org/SiteCollectionDocuments/ScreeningTool.pdf).

Although knowledge gained from the SSC Phase III program led to improvements in treating severe sepsis, ward patients continued to have poor outcomes. To address the potential contributions of delayed case identification, we developed an early recognition and treatment program. We outline the steps we took to develop this multisite PI program.

PREPARATORY WORK

During the planning phase, several procedural steps were taken before initiating the ward sepsis program (Table 3). These required 3 levels of involvement: senior administration, midlevel management, and patient‐level support.

Table 3.Critical Steps Prior to Initiating a Ward Sepsis‐Detection Program
NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit.
1.	Obtain administrative support (ie, funding for data collection, project lead, informatics)
2.	Align with ED and ICU
3.	Identify 1 ward to pilot the program
4.	Establish unit‐based champions on each shift (nurse, physician)
5.	Review ward workflow
6.	Develop nurse screening tool
7.	Provide education

Administrative Support

In the course of our implementation effort, we found that sites that had high‐level administrative support were more likely to implement and sustain the intervention. For this reason, we consider such support to be critical. Examples of such support include chief medical officers, chief nursing officers, and chief quality officers. As an example, securing commitment from hospital leadership may be necessary to improve/change the EHR and provide funding for project management to achieve sustainable improvement in outcomes. Aligning leadership with frontline physicians, nurses, and support staff toward a common goal provides the platform for a successful program.[11]

ED and ICU Leadership Support

Maintaining lines of communication among the ED, ICU, and ward staff is critical to improving outcomes. Establishing a cohesive system (ED, ICU, and wards) aimed at early recognition and treatment of sepsis throughout the hospital stay can lead to improvement in continuity of care and outcomes. For example, when an ED severe sepsis patient is transferred to the ward and subsequently requires admission to the ICU due to declining clinical status, providing timely feedback to the ED can help improve care for subsequent patients. Collaboration between the ED and the ward can also contribute to improved transitions of care for patients with severe sepsis.

Hospitalist/Internal Medicine Leadership

Our experience with implementing sepsis bundles in the ED and ICU highlights the need for effective interdisciplinary collaboration with designated physician and nurse leaders/champions. We found that engaging local clinical leaders in the early recognition and management of a severe sepsis QI program is imperative for the program's success. Hospitalists are often the physician leaders for the inpatient wards, so it is essential to secure their early engagement, support, and leadership. Moreover, though collaboration with ED and ICU physicians may be useful, as described above, a hospitalist champion is likely to be more effective at educating other hospitalists about the program, overcoming physician resistance, and facilitating change.

Depending on a hospital's size and workflows, designated ward‐ or shift‐based hospitalists and nurses as champions can serve as key resources to support implementation. These individuals help establish mutual respect and a common mental model of how sepsis can evolve in ward patients. Even more important, by providing assistance with both the screening tool as well as with recognition itself, these individuals not only speed implementation, but also protect against rough patches (ie, those instances where workflow changes run into resistance).

EDUCATION

Diagnosing sepsis is not always easy, making education on sepsis recognition, evaluation, and treatment necessary prior to implementation. Retention of knowledge over time through review and refresher courses are methods we used in the program. Providing background material explaining why education is necessary and providing physicians and nurses with materials to help them recall the information over time were developed at several sites. Resources included sepsis posters, identification‐size badge cards with the sepsis bundle elements, and bulletin boards on the wards with information to reinforce sepsis recognition, evaluation, and treatment. Education for the ward‐centric program included an overview of the SSC guidelines, supportive literature, sepsis definitions, description of the infection's systemic manifestations, criteria for identification of new‐ onset organ dysfunction, and the details on current severe sepsis 3‐ and 6‐hour bundle requirements. We made clinicians aware of resources available on the SSC website.[12] Data emphasizing the incidence of sepsis, as well as outcomes and motives for the QI wards program, were incorporated during the collaborative meetings. Data can serve as strong motivators for action (eg, highlighting current incidence rates). Many hospitals combined presentation of these aggregate data with local review of selected cases of severe sepsis that occurred in their own wards.

Understanding that the training for and experiences of ED, ICU, and ward nurses varies, nurse education contained critical assessment skills in determining when to suspect a new or worsening infection. Training nurses to complete a comprehensive daily infection assessment may help them overcome uncertainty in judgement. Assessment skills include examination of invasive lines, surgical sites, wounds, and presence of a productive cough. Equally important, patients being treated for an infection would benefit from a daily assessment for improvement or worsening of the infection. Information uncovered may identify early signs of organ failure in addition to infections that may need further evaluation and treatment. Education provides knowledge, but achieving program success relies heavily on staff accepting that they can make a difference in sepsis patient identification, management, and outcomes.

SCREENING METHODS, COMMUNICATION, AND PROTOCOLS

The SSC tool for severe sepsis facilitates screening for (1) confirmed or suspected infection, (2) presence of 2 or more systemic manifestations of infection, and (3) acute organ dysfunction. This tool was the basis for the do (screening) portion of the PDSA model.

Continuous Screening

Technology can facilitate early recognition of severe sepsis with EHR‐based surveillance screening tools. Surveillance may include continuous review of vital signs and laboratory values with an automated alerting system. A valuable feature of the screening tool alert is the incorporation of the nurse's assessment. Decision support can improve the process by providing advice with systems requiring a reason to over‐ride the advice.[13] For example, an alert may include input from the nurse to determine if the abnormal data are thought to be related to an infectious process or due to another cause. If a suspected or confirmed infection is identified, further surveillance screening can include review of blood pressure readings and laboratory data to determine if organ dysfunction is present. If organ dysfunction criteria are identified, the alert can prompt the nurse to notify the physician to discuss whether the organ dysfunction is new and related to the infection and if implementation of the severe sepsis bundles is indicated (Figure 2). Additional continuous screening models may include variations of the example provided to include alerts to other clinicians or a response team.

Severe sepsis alert with situation, background, assessment, recommendation (SBAR) embedded. Abbreviations: BMP, basic metabolic panel; BP, blood pressure; CBC, complete blood count; INR, International Normalized Ratio; IV, intravenous; PTT, partial thromboplastin time; SIRS, systemic inflammatory response syndrome; SpO2, saturation of peripheral oxygen; WBC, white blood cells.

An automated screening tool within the EHR can be useful because the system continuously scans to identify signs and symptoms of sepsis, thus providing screening consistency, and offers data on the back end to be used as a mechanism for feedback to monitor effectiveness. Challenges with EHR severe sepsis alert development are resource allocation, testing, education, and ongoing evaluation and feedback. Other challenges include the potential for alert fatigue (false positive) and inappropriate response (false negative) to the infection prompt, thereby halting the next step in automated screening for organ dysfunction. Time to complete an automated screening tool varies based on strategic design and user understanding.

Screening Checklist

Whereas EHR tools may be effective in early recognition of sepsis, not all sites will have the capability to use these tools because of lack of informatics support, cost of development, and absence of an EHR in some hospitals.[14] An alternative to continuous screening is a sepsis checklist such as the severe sepsis screening tool (Figure 1). The checklist is designed to prompt nurses to screen every patient during every shift for new signs of sepsis and organ dysfunction.

The checklist ensures that 3 key issues are considered: presence of a suspected or confirmed infection, systemic manifestations of inflammation, and physiological manifestations of organ dysfunction. The paper tool is simple to use and can be completed in 10 to 20 minutes. It requires the nurse to review the progress notes, vital signs, and laboratory test results. Although the time investment seems onerous, the gain in consistency of screening and treatment compensates for the extra effort. Review of the checklist also provides a locus for feedback and new improvement cycles.

Scripted Communication

Once a patient with severe sepsis is identified, communicating this finding to the rest of the clinical team is essential. Because communication skills are not always emphasized in QI projects, we decided to emphasize a structured approach. We provided clinicians with scripts based on the SBAR (situation, background, assessment, and recommendation) technique aimed to improve communication (Figure 3).[15, 16] Using the SBAR technique also supports our efforts to build nurses' confidence and willingness to employ protocols that give them greater autonomy.

Script for communicating severe sepsis. Abbreviations: CBC = complete blood count; WBC, white blood cells.

Nurse‐Directed Protocols

Skillful identification and management of severe sepsis patients constitute the foundation for implementation of nurse‐directed protocols in this patient population. Such protocols promote autonomy and staff ownership. Severe sepsis protocols may include increasing the frequency of vital signs, placement of laboratory orders and, in sites with an established culture of increased nurse autonomy, initiation of intravenous access and a fluid bolus when specific criteria are met. Because nursing scope of practice varies from state to state and among hospitals, nurse‐directed severe sepsis protocols generally require review of current site practice guidelines, physician agreement, and approval by the medical executive committee prior to implementation. Despite these differences, maximizing nurse leadership involvement and nurse autonomy can help propel the program forward. Protocols may be implemented based on knowledge level and resources on a particular ward. A workflow evaluation may be included in this process to define staff performing each step, what is being reported, and where and when data are recorded.

DATA COLLECTION AND FEEDBACK

Nurse screening drives the ward program and ensuring its consistency is the key to early patient identification. We made ongoing repeated evaluation of the appropriate use of the screening tool, time to physician notification, and time to follow‐up intervention, a critical part of the study phase of the PDSA cycle. Once the nursing staff is consistently accurate and compliant (>90%) with screening, random (eg, once per week) screening tool review may be more suitable, thus requiring fewer resources (see Supporting Information, Appendix 1, in the online version of this article).

Data Collection

A key to improvement is to study the process, which requires data collection to assess compliance. In our experience, timely clinician feedback, along with data, led to effective process change. Real‐time data collection and discussion with the clinical team may lead to early recognition or intervention.

In our collaborative experience, we observed varied resources and timing for data collection across hospitals. For example, several participating sites had sepsis coordinators to collect data, whereas others relied on the quality department or nursing staff to collect data. Data may be collected concurrently (within 24 hours of severe sepsis presentation) or retrospectively. Retrospective data collection may allow for staff flexibility in data collection, but limits feedback to the clinicians. For example, with retrospective review, early recognition and treatment failure may go unrecognized until the data are analyzed and reported, which can be months after the patient has been discharged or expired.

Feedback to Caregivers

A consistent feedback process, which can occur at the individual or group level, may lead to prompt improvement in severe sepsis management. An example of individual feedback would be providing the nurse with the elapsed time from antibiotic order to time of administration. Early in the implementation phase, frequent (daily or weekly) feedback is helpful to build team cohesiveness. An example of feedback to build the team may include a unit‐based report on the last 5 severe sepsis patients managed by the group. Providing overall bundle compliance and outcome reports on a weekly and monthly basis will allow the clinical team to track progress. Examples of report cards and a dashboard are provided in the supplemental material, which highlight compliance with the bundle elements as well as time to achieve the bundle elements. (see Supporting Information, Appendix 2 and Appendix 3, in the online version of this article). Resources to evaluate and provide consistent data may require up to 10 to 15 hours per week for 1 unit. Automated reports may decrease the resources needed in collating and reporting data.

OUTCOME MEASURES

Although certainly important, mortality is not the only outcome measure worthy of measurement. Other relevant outcomes include transfers to a higher level of care and need for major supportive therapies (eg, dialysis, mechanical ventilation, vasopressor infusion). Whereas it is valuable to review transfers to a higher level of care, we emphasized that these are not necessarily adverse outcomes; in fact, in many cases such transfers are highly desirable. It is also important to track the overall impact of sepsis on hospital length of stay.

SUMMARY/CONCLUSIONS

Grounded in the Institute for Healthcare Improvement's PDSA QI model, we developed a program aimed at improving outcomes for severe sepsis ward patients. Our program's cornerstone is nurse‐led checklist‐based screening. Our faculty led learning sessions that concentrated on using a collaborative approach whose key components were education in early sepsis identification, use of a sepsis screening tool, and the SBAR method for effective communication. Pitfalls identified during the program included lack of knowledge for both nurses and physicians in early severe sepsis identification, resistance to routine screening, and lack of data collection and leadership support. The most successful participating sites were those with senior leadership backing, staff engagement, informatics support, and data collection resources. Ultimately, replicating a program such as ours will depend on team cohesiveness, and nurse empowerment through the use of nurse‐driven protocols. Programs like this may lead to progression toward standardizing practice (eg, antibiotic administration, fluid resuscitation), matching patient needs to resources, and building stronger partnerships between hospitalists and nurses.

Disclosures

This work was supported by a grant provided to the Society of Critical Care Medicine by the Gordon and Betty Moore Foundation (Early Identification and Management of Sepsis on the Wards). The work was supported by a grant from the Adventist Hospital System. As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Moore Foundation played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component; the same was the case with the other sponsors. The authors report no conflicts of interest.

Files

References

Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Intensive Care Med. 2010;36(2):222–231.
Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5‐year study. Intensive Care Med. 2014;40(11):1623–1633.
Rohde JM, Odden AJ, Bonham C, et al. The epidemiology of acute organ system dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med. 2013;8(5):243–247.
Yealy DM, Huang DT, Delaney A, et al. Recognizing and managing sepsis: what needs to be done? BMC Med. 2015;13:98.
Sopena N, Heras E, Casas I, et al. Risk factors for hospital‐acquired pneumonia outside the intensive care unit: a case‐control study. Am J Infect Control. 2014;42(1):38–42.
Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Crit Care Med. 2013;41(2):580–637.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nembhard IM. Learning and improving in quality improvement collaboratives: which collaborative features do participants value most? Health Serv Res. 2009;44(2 pt 1):359–378.
Pronovost PJ, Weast B, Bishop K, et al. Senior executive adopt‐a‐work unit: a model for safety improvement. Jt Comm J Qual Saf. 2004;30(2):59–68.
Surviving Sepsis Campaign. Available at: http://survivingsepsis.org/Resources/Pages/default.aspx. Accessed September 24, 2015.
Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta‐regression of 162 randomised trials. BMJ. 2013;346:f657.
Bhounsule P, Peterson AM. characteristics of hospitals associated with complete and partial implementation of electronic health records. Perspect Health Inf Manag. 2016;13:1c.
Institute for Healthcare Improvement. SBAR technique for communication: a situational briefing model. Available at: http://www.ihi.org/resources/pages/tools/sbartechniqueforcommunicationasituationalbriefingmodel.aspx. Accessed September 12, 2015.
Compton J, Copeland K, Flanders S, et al. Implementing SBAR across a large multihospital health system. Jt Comm J Qual Patient Saf. 2012;38(6):261–268.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S32-S39

Read more about Program for Early Detection of Sepsis

Sections

Author(s)

Andrew Odden, MD, SFHM

Laura Evans, MD, MSc

Snehal Gandhi, MD

Sean Townsend, MD

Mitchell Levy, MD

Author(s)

Andrew Odden, MD, SFHM

Laura Evans, MD, MSc

Snehal Gandhi, MD

Sean Townsend, MD

Mitchell Levy, MD

Files

Files

Article PDF

Article PDF

Table 1.Presentation of Severe Sepsis in the Emergency Department and the Ward
	Emergency Department Presentation	Ward Presentation
Patient‐familyreported symptoms	I just feel sick, family reports disorientation, not eating	Currently hospitalized, family often not present, diagnosis may not be clear, baseline mental status unknown, lack of appetite may be linked to dislike of hospital food.
Systemic manifestations	Triage observed 2 or more signs of infection or patient reports temperature while at home plus additional finding on assessment.	Signs of infection may appear 1 at a time, hours apart, and may appear to be mild changes to staff or missed entirely due to staff discontinuity.
Organ dysfunction	Present on admission; triage nurse assesses for organ dysfunction.	Develops over hours or days; may be subtle or acute.
Laboratory study process	Ordered and evaluated within 1 hour.	Not routinely completed daily, may be ordered after physician evaluation or during rounds. Results within 34 hours.

Table 2.Surviving Sepsis Campaign 3‐Hour Severe Sepsis Bundle
To be completed within 3 hours of time of presentation
1. Measure lactate level
2. Obtain blood cultures prior to administration of antibiotics
3. Administer broad‐spectrum antibiotics
4. Administer 30 mL/kg crystalloid for hypotension or lactate 4 mmol/L (36 mg/dL)

PREPARATORY WORK

Table 3.Critical Steps Prior to Initiating a Ward Sepsis‐Detection Program
NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit.
1.	Obtain administrative support (ie, funding for data collection, project lead, informatics)
2.	Align with ED and ICU
3.	Identify 1 ward to pilot the program
4.	Establish unit‐based champions on each shift (nurse, physician)
5.	Review ward workflow
6.	Develop nurse screening tool
7.	Provide education

Administrative Support

ED and ICU Leadership Support

Hospitalist/Internal Medicine Leadership

EDUCATION

SCREENING METHODS, COMMUNICATION, AND PROTOCOLS

Continuous Screening

Screening Checklist

Scripted Communication

Nurse‐Directed Protocols

DATA COLLECTION AND FEEDBACK

Data Collection

Feedback to Caregivers

OUTCOME MEASURES

SUMMARY/CONCLUSIONS

Disclosures

Table 1.Presentation of Severe Sepsis in the Emergency Department and the Ward
	Emergency Department Presentation	Ward Presentation
Patient‐familyreported symptoms	I just feel sick, family reports disorientation, not eating	Currently hospitalized, family often not present, diagnosis may not be clear, baseline mental status unknown, lack of appetite may be linked to dislike of hospital food.
Systemic manifestations	Triage observed 2 or more signs of infection or patient reports temperature while at home plus additional finding on assessment.	Signs of infection may appear 1 at a time, hours apart, and may appear to be mild changes to staff or missed entirely due to staff discontinuity.
Organ dysfunction	Present on admission; triage nurse assesses for organ dysfunction.	Develops over hours or days; may be subtle or acute.
Laboratory study process	Ordered and evaluated within 1 hour.	Not routinely completed daily, may be ordered after physician evaluation or during rounds. Results within 34 hours.

Table 2.Surviving Sepsis Campaign 3‐Hour Severe Sepsis Bundle
To be completed within 3 hours of time of presentation
1. Measure lactate level
2. Obtain blood cultures prior to administration of antibiotics
3. Administer broad‐spectrum antibiotics
4. Administer 30 mL/kg crystalloid for hypotension or lactate 4 mmol/L (36 mg/dL)

PREPARATORY WORK

Table 3.Critical Steps Prior to Initiating a Ward Sepsis‐Detection Program
NOTE: Abbreviations: ED, emergency department; ICU, intensive care unit.
1.	Obtain administrative support (ie, funding for data collection, project lead, informatics)
2.	Align with ED and ICU
3.	Identify 1 ward to pilot the program
4.	Establish unit‐based champions on each shift (nurse, physician)
5.	Review ward workflow
6.	Develop nurse screening tool
7.	Provide education

Administrative Support

ED and ICU Leadership Support

Hospitalist/Internal Medicine Leadership

EDUCATION

SCREENING METHODS, COMMUNICATION, AND PROTOCOLS

Continuous Screening

Screening Checklist

Scripted Communication

Nurse‐Directed Protocols

DATA COLLECTION AND FEEDBACK

Data Collection

Feedback to Caregivers

OUTCOME MEASURES

SUMMARY/CONCLUSIONS

Disclosures

References

Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Intensive Care Med. 2010;36(2):222–231.
Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5‐year study. Intensive Care Med. 2014;40(11):1623–1633.
Rohde JM, Odden AJ, Bonham C, et al. The epidemiology of acute organ system dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med. 2013;8(5):243–247.
Yealy DM, Huang DT, Delaney A, et al. Recognizing and managing sepsis: what needs to be done? BMC Med. 2015;13:98.
Sopena N, Heras E, Casas I, et al. Risk factors for hospital‐acquired pneumonia outside the intensive care unit: a case‐control study. Am J Infect Control. 2014;42(1):38–42.
Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Crit Care Med. 2013;41(2):580–637.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nembhard IM. Learning and improving in quality improvement collaboratives: which collaborative features do participants value most? Health Serv Res. 2009;44(2 pt 1):359–378.
Pronovost PJ, Weast B, Bishop K, et al. Senior executive adopt‐a‐work unit: a model for safety improvement. Jt Comm J Qual Saf. 2004;30(2):59–68.
Surviving Sepsis Campaign. Available at: http://survivingsepsis.org/Resources/Pages/default.aspx. Accessed September 24, 2015.
Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta‐regression of 162 randomised trials. BMJ. 2013;346:f657.
Bhounsule P, Peterson AM. characteristics of hospitals associated with complete and partial implementation of electronic health records. Perspect Health Inf Manag. 2016;13:1c.
Institute for Healthcare Improvement. SBAR technique for communication: a situational briefing model. Available at: http://www.ihi.org/resources/pages/tools/sbartechniqueforcommunicationasituationalbriefingmodel.aspx. Accessed September 12, 2015.
Compton J, Copeland K, Flanders S, et al. Implementing SBAR across a large multihospital health system. Jt Comm J Qual Patient Saf. 2012;38(6):261–268.

References

Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Intensive Care Med. 2010;36(2):222–231.
Levy MM, Dellinger RP, Townsend SR, et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
Levy MM, Rhodes A, Phillips GS, et al. Surviving Sepsis Campaign: association between performance metrics and outcomes in a 7.5‐year study. Intensive Care Med. 2014;40(11):1623–1633.
Rohde JM, Odden AJ, Bonham C, et al. The epidemiology of acute organ system dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med. 2013;8(5):243–247.
Yealy DM, Huang DT, Delaney A, et al. Recognizing and managing sepsis: what needs to be done? BMC Med. 2015;13:98.
Sopena N, Heras E, Casas I, et al. Risk factors for hospital‐acquired pneumonia outside the intensive care unit: a case‐control study. Am J Infect Control. 2014;42(1):38–42.
Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign: international guidelines for management of severe sepsis and septic shock, 2012. Crit Care Med. 2013;41(2):580–637.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nembhard IM. Learning and improving in quality improvement collaboratives: which collaborative features do participants value most? Health Serv Res. 2009;44(2 pt 1):359–378.
Pronovost PJ, Weast B, Bishop K, et al. Senior executive adopt‐a‐work unit: a model for safety improvement. Jt Comm J Qual Saf. 2004;30(2):59–68.
Surviving Sepsis Campaign. Available at: http://survivingsepsis.org/Resources/Pages/default.aspx. Accessed September 24, 2015.
Roshanov PS, Fernandes N, Wilczynski JM, et al. Features of effective computerised clinical decision support systems: meta‐regression of 162 randomised trials. BMJ. 2013;346:f657.
Bhounsule P, Peterson AM. characteristics of hospitals associated with complete and partial implementation of electronic health records. Perspect Health Inf Manag. 2016;13:1c.
Institute for Healthcare Improvement. SBAR technique for communication: a situational briefing model. Available at: http://www.ihi.org/resources/pages/tools/sbartechniqueforcommunicationasituationalbriefingmodel.aspx. Accessed September 12, 2015.
Compton J, Copeland K, Flanders S, et al. Implementing SBAR across a large multihospital health system. Jt Comm J Qual Patient Saf. 2012;38(6):261–268.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S32-S39

Page Number

S32-S39

Publications

Publications

Article Type

Display Headline

Implementation of a multicenter performance improvement program for early detection and treatment of severe sepsis in general medical–surgical wards

Display Headline

Implementation of a multicenter performance improvement program for early detection and treatment of severe sepsis in general medical–surgical wards

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Christa Schorr, Cooper Research Institute–Critical Care, Cooper University Hospital, One Cooper Plaza, Dorrance Building, Suite 411, Camden, NJ 08103; Telephone: 856‐968‐7493; Fax: 856‐968‐8378; E‐mail: schorr-christa@cooperhealth.edu

Content Gating

Gated (full article locked unless allowed per User)

Alternative CME

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

Media Files

Critical Illness Outside the ICU

Article Type

Changed

Mon, 01/02/2017 - 19:34

Display Headline

Early detection, prevention, and mitigation of critical illness outside intensive care settings

Author(s)

R. Phillip Dellinger, MD

This issue of the Journal of Hospital Medicine describes 2 research and quality improvement demonstration projects funded by the Gordon and Betty Moore Foundation. Early detection is central to both projects. This introductory article does not provide a global review of the now voluminous literature on rapid response teams (RRTs), sepsis detection systems, or treatment protocols. Rather, it takes a step back and reassesses just what early detection and quantification of critical illness are. It then examines the implications of early detection and its quantification.

CONCEPTUAL FRAMEWORK

We define severe illness as the presence of acute disease such that a person can no longer expect to improve without dedicated hospital treatment but which is not inevitably associated with mortality, postdischarge morbidity, or major loss of autonomy. In contrast, we define critical illness as acute disease with high a priori risk of mortality, postdischarge morbidity, and major (possibly total) loss of autonomy. We accept that the boundaries between ordinary illness, severe illness, and critical illness are blurred. The basic assumption behind all efforts at early detection is that these edges can be made sharp, and that the knowledge base required to do so can also lead to improvements in treatment protocols and patient outcomes. Further, it is assumed that at least some forms of critical illness can be prevented or mitigated by earlier detection, identification, and treatment.

Research over the last 2 decades has provided important support for this intuitive view as well as making it more nuanced. With respect to epidemiology, the big news is that sepsis is the biggest culprit, and that it accounts for a substantial proportion of all hospital deaths, including many previously considered unexpected hospital deaths due to in‐hospital deterioration.[1] With respect to treatment, a number of studies have demonstrated that crucial therapies previously considered to be intensive care unit (ICU) therapies can be initiated in the emergency department or general medicalsurgical ward.[2]

Figure 1 shows an idealized framework for illness presenting in the emergency department or general medicalsurgical wards. It illustrates the notion that a transition period exists when patients may be rescued with less intense therapy than will be required when condition progression occurs. Once a certain threshold is crossed, the risk of death or major postdischarge morbidity rises exponentially. Unaided human cognition's ability to determine where a given patient is in this continuum is dangerously variable and is highly dependent on the individuals training and experience. Consequently, as described in several of the articles in this issue as well as multiple other publications, health systems are employing comprehensive electronic medical records (EMRs) and are migrating to algorithmic approaches that combine multiple types of patient data.[3, 4] Although we are still some distance from being able to define exact boundaries between illness, severe illness, and critical illness, current EMRs permit much better definition of patient states, care processes, and short‐term outcomes.

Relationship between time, course of illness (solid line), risk of death or major disability (dashed line), and possible detection periods among patients who present in the emergency department or general medical–surgical ward. All axes employ hypothetical units, because empiric data are not currently available for all domains listed. Point C represents when unaided human cognition (ordinary clinical judgment) can first detect incipient deterioration. In theory, algorithmic approaches (point A) based on real‐time data from the electronic medical record (EMR) can provide earlier detection, and novel biomarkers (point B) could lead to even earlier detection.

Whereas our ability to quantify many processes and short‐term outcomes is expanding rapidly, quantification of the possible benefit of early detection is complicated by the fact that, even in the best of circumstances, not all patients can be rescued. For some patients, rescue may be temporary, raising the prospect of repeated episodes of critical illness and prolonged intensive care without any hope of leaving the hospital. Figure 2 shows that, for these patients, the problem is no longer simply one of preventing death and preserving function but, rather, preserving autonomy and dignity. In this context, early detection means earlier specification of patient preferences.[5, 6]

Progression to critical illness among patients near the end of life. Given that it may not be possible to prevent death, what matters most to patients and families is preservation of autonomy and ability to make choices concordant with their values and preferences. In theory, early detection combined with appropriate palliative care could maximize preservation of autonomy (upper arrow), whereas, in their absence, the health system enters the current default mode (lower arrow) in which intensive care is initiated despite low likelihood of preventing death or disability.

JUST WHAT CONSTITUTES EARLY DETECTION (AND HOW DO WE QUANTIFY IT)?

RRTs arose as the result of a number of studies showing thatin retrospectin‐hospital deteriorations should not have been unexpected. Given comprehensive inpatient EMRs, it is now possible to develop more rigorous definitions. A minimum set of parameters that one would need to specify for proper quantification of early detection is shown on Figure 3. The first is specifying a T₀, that is, the moment when a prediction regarding event X (which needs to be defined) is issued. This is different from the (currently unmeasurable) biologic onset of illness as well as the first documented indication that critical illness was present. Further, it is important to be explicit about the event time frame (the time period during which a predicted event is expected to occur): we are predicting that X will occur within E hours of the T₀. The time frame between the T₀ and X, which we are referring to as lead time, is clinically very important, as it represents the time period during which the response arm (eg, RRT intervention) is to be instituted. Statistical approaches can be used to estimate it, but once an early detection system is in place, it can be quantified. Figure 3 is not restricted to electronic systems; all components shown can be and are used by unaided human cognition.

Characterizing early warning systems. At a T0, a detection system issues a probability estimate that an undesirable event, X (which must be defined explicitly) will occur within some elapsed time (point E) (EVENT TIME FRAME). Time required for a response arm to prepare an intervention is LEAD TIME. Development of detection systems is complicated by the fact that the time point when biological critical illness actually begins is currently unmeasurable, whereas system development is limited by how accurately X is documented. Probability estimates are based on data sources with different accumulation times. Some definitional data elements (eg, age, gender, diagnosis for this admission) are not recurrent (♦). Others, which could include streaming data, are recurrent, and the look‐back time frame must be clearly specified. For example, physiologic or biochemical data generally accumulate over a short time period (usually measured in hours); health services data (eg, elapsed length of stay in the hospital at T0; was this patient recently in the intensive care unit?) are typically measured in days, whereas chronic conditions can be measured in months to years.

Impact of patients with restricted resuscitation status (not full code, which includes partial code, do not resuscitate, and comfort care only) on unplanned transfers to the intensive care unit (ICU) and total 30‐day mortality. Data are from 21 Kaiser Permanente Northern California hospitals between May 1, 2012 and October 31, 2013. The left panels show patients with restricted resuscitation status (12.1% of patients; range across hospitals, 6.5% to 18.0%), who accounted for 53% of all deaths. Full code patients directly admitted to the ICU and all other hospital units are shown in the middle and right panels, respectively. Circles are drawn to scale (proportion of admissions in top panels, proportion of deaths in lower panels). Within each circle, the shaded area represents the proportion of patients who experienced unplanned transfer to intensive care (for direct ICU admits, this refers to return transfers to the ICU after discharge from the ICU).

It is essential to specify what data are used to generate probability estimates as well as the time frames used, which we refer to as the look‐back time frames. Several types of data could be employed, with some data elements (eg, age or gender) being discrete data with a 1:1 fixed correspondence between the patient and the data. Other data have a many‐to‐1 relationship, and an exact look‐back time frame must be specified for each data type. For example, it seems reasonable to specify a short (1224 hours) look‐back period for some types of data (eg, vital signs, lactate, admission diagnosis or chief complaint), an intermediate time period (13 days) for information on the current encounter, and a longer (months to years) time period for preexisting illness or comorbidity burden.

Because many events are rare, traditional measures used to assess model performance, such as the area under the receiver operator characteristic curve (C statistic), are not as helpful.[7] Consequently, much more emphasis needs to be given to 2 key metrics: number needed to evaluate (or workup to detection ratio) and threshold‐specific sensitivity (ability of the alert to detect X at a given threshold). With these, one can answer 3 questions that will be asked by the physicians and nurses who are not likely to be researchers, and who will have little interest in the statistics: How many patients do I need to work up each day? How many patients will I need to work up for each possible outcome identified? For this amount of work, how many of the possible outcomes will we catch?

Data availability for the study of severe and critical illness continues to expand. Practically, this means that future research will require more nuanced ontologies for the classification of physiologic derangement. Current approaches to severity scoring (collapsing data into composite scores) need to be replaced by dynamic approaches that consider differential effects on organ systems as well as what can be measured. Severity scoring will also need to incorporate the rate of change of a score (or probability derived from a score) in predicting the occurrence of an event of interest as well as judging response to treatment. Thus, instead of at time of ICU admission, the patient had a severity score of 76, we may have although this patient's severity score at the time of admission was decreasing by 4 points per hour per 10 mL/kg fluid given, the probability for respiratory instability was increasing by 2.3% per hour given 3 L/min supplemental oxygen. This approach is concordant with work done in other clinical settings (eg, in addition to an absolute value of maximal negative inspiratory pressure or vital capacity, the rate of deterioration of neuromuscular weakness in Guillain‐Barr syndrome is also important in predicting respiratory failure[8]).

Electronic data also could permit better definition of patient preferences regarding escalation of care. At present, available electronic data are limited (primarily, orders such as do not resuscitate).[9] However, this EMR domain is gradually expanding.[10, 11] Entities such as the National Institutes of Health could develop sophisticated and rapid questionnaires around patient preferences that are similar to those developed for the Patient Reported Outcomes Measurement Information System.[12] Such tools could have a significant effect on our ability to quantify the benefits of early detection as it relates to a patient's preferences (including better delineation of what treatments they would and would not want).

ACTIVATING A RESPONSE ARM

Early identification, antibiotic administration, fluid resuscitation, and source control are now widely felt to constitute low‐hanging fruit for decreasing morbidity and mortality in severe sepsis. All these measures are included in quality improvement programs and sepsis bundles.[13, 14, 15] However, before early interventions can be instituted, sepsis must at least be suspected, hence the need for early detection. The situation with respect to patient deterioration (for reasons other than sepsis) in general medical surgical wards is less clear‐cut. Reasons for deterioration are much more heterogenous and, consequently, early detection is likely necessary but not sufficient for outcomes improvement.

The 2 projects described in this issue describe nonspecific (indicating elevated risk but not specifying what led to the elevation of risk) and sepsis‐specific alerting systems. In the case of the nonspecific system, detection may not lead to an immediate deployment of a response arm. Instead, a secondary evaluation process must be triggered first. Following this evaluation component, a response arm may or may not be required. In contrast, the sepsis‐specific project essentially transforms the general medicalsurgical ward into a screening system. This screening system then also triggers specific bundle components.

Neither of these systems relies on unaided human cognition. In the case of the nonspecific system, a complex equation generates a probability that is displayed in the EMR, with protocols specifying what actions are to be taken when that probability exceeds a prespecified threshold. With respect to the sepsis screening system, clinicians are supported by EMR alerts as well as protocols that increase nursing autonomy when sepsis is suspected.

The distinction between nonspecific (eg, acute respiratory failure or hemodynamic deterioration) and specific (eg, severe sepsis) alerting systems is likely to disappear as advances in the field occur. For example, incorporation of natural language processing would permit inclusion of semantic data, which could be processed so as to prebucket an alert into one that not just gave a probability, but also a likely cause for the elevated probability.

In addition, both types of systems suffer from the limitation of working off a limited database because, in general, current textbooks and training programs primary focus remains that of treatment of full‐blown clinical syndromes. For example, little is known about how one should manage patients with intermediate lactate values, despite evidence showing that a significant percentage of patients who die from sepsis will initially have such values, with 1 study showing 63% as many deaths with initial lactate of 2.5 to 4.0 mmol/L as occurred with an initial lactate of >4.0 mmol/L.[16] Lastly, as is discussed below, both systems will encounter similar problems when it comes to quantifying benefit.

QUANTIFYING BENEFIT

Whereas the notion of deploying RRTs has clearly been successful, success in demonstrating unequivocal benefit remains elusive.[17, 18, 19] Outcome measures vary dramatically across studies and have included the number of RRT calls, decreases in code blue events on the ward, and decreases in inpatient mortality.[20] We suspect that other reasons are behind this problem. First is the lack of adequate risk adjustment and ignoring the impact of patients near the end of life on the denominator. Figure 4 shows recent data from 21 Kaiser Permanente Northern California (KPNC) hospitals, which can now capture care directive orders electronically,[21] illustrates this problem. The majority (53%) of hospital deaths occur among a highly variable proportion (range across hospitals, 6.5%18.0%) of patients who arrive at the hospital with a restricted resuscitation preference (do not resuscitate, partial code, and comfort care only). These patients do not want to die or crash and burn but, were they to trigger an alert, they would not necessarily want to be rescued by being transferred to the ICU either; moreover, internal KPNC analyses show that large numbers of these patients have sepsis and refuse aggressive treatment. The second major confounder is that ICUs save lives. Consequently, although early detection could lead to fewer transfers to the ICU, using the end point of ICU admission is very problematic, because in many cases the goal of alerting systems should be to get patients to the ICU sooner, which would not affect the outcome of transfer to the ICU in a downward direction; in fact, such systems might increase transfer to the ICU.

The complexities summarized in Figure 4 mean that it is likely that formal quantification of benefit will require examination of multiple measures, including balancing measures as described below. It is also evident that, in this respectlack of agreement as to what constitutes a good outcomethe issues being faced here are a reflection of a broader area of disagreement within our profession and society at large that extends to medical conditions other than critical illness.

POTENTIAL HARMS OF EARLY DETECTION

Implementation of early detection and rapid response systems are not inherently free of harm. If these systems are not shown to have benefit, then the cost of operating them is moving resources away from other, possibly evidence‐based, interventions.[22] At the individual level, alerts could frighten patients and their families (for example, some people are very uncomfortable with the idea that one can predict events). Physicians and nurses who work in the hospital are already quite busy, so every time an alert is issued, it adds to the demand on their already limited time, hence, the critical importance of strategies to minimize false alarms and alert fatigue. Moreover, altering existing workflows can be disruptive and unpopular.

A potentially more quantifiable problem is the impact of early detection systems on ICU operations. For example, if an RRT decides to transfer a patient from the ward to the ICU as a preventive measure (soft landing) and this in turn ties up an ICU bed, that bed is then unavailable for a new patient in the emergency department. Similarly, early detection systems coupled with structured protocols for promoting soft landings could result in a change in ICU case mix, with greater patient flow due to increased numbers of patients with lower severity and lower ICU length of stay. These considerations suggest the need to couple early detection with other supportive data systems and workflows (eg, systems that monitor bed capacity proactively).

Lastly, if documentation protocols are not established and followed, early detection systems could expose both individual clinicians as well as healthcare institutions to medicallegal risk. This consideration could be particularly important in those instances where an alert is issued and, for whatever reasons, clinicians do not take action and do not document that decision. At present, early detection systems are relatively uncommon, but they may gradually become standard of care. This means that in‐house out of ICU deteriorations, which are generally considered to be bad luck or due to a specific error or oversight, may then be considered to be preventable. Another possible scenario that could arise is that of plaintiffs invoking enterprise liability, where a hospital's not having an early detection system becomes considered negligent.

ARTICLES IN THIS ISSUE

In this issue of the Journal of Hospital Medicine, we examine early detection from various perspectives but around a common theme that usually gets less attention in the academic literature: implementation. The article by Schorr et al.[23] describes a disease‐specific approach that can be instantiated using either electronic or paper tools. Escobar et al.[24] describe the quantitative as well as the electronic architecture of an early warning system (EWS) pilot at 2 hospitals that are part of an integrated healthcare delivery system. Dummett et al.[25] then show how a clinical rescue component was developed to take advantage of the EWS, whereas Granich et al.[26] describe the complementary component (integration of supportive care and ensuring that patient preferences are respected). The paper by Liu et al.[27] concludes by placing all of this work in a much broader context, that of the learning healthcare system.

FUTURE DIRECTIONS: KEY GAPS IN THE FIELD

Important gaps remain with respect to early detection and response systems. Future research will need to focus on a number of areas. First and foremost, better approaches to quantifying the costbenefit relationships of these systems are needed; somehow, we need to move beyond a purely intuitive sense that they are good things. Related to this is the need to establish metrics that would permit rigorous comparisons between different approaches; this work needs to go beyond simple comparisons of the statistical characteristics of different predictive models. Ideally, it should include comparisons of different approaches for the response arms as well. We also need to characterize clinician understanding about detection systems, what constitutes impending or incipient critical illness, and the optimum way to provide early detection. Finally, better approaches to integrating health services research with basic science work must be developed; for example, how should one test new biomarkers in settings with early detection and response systems?

The most important frontier, however, is how one can make early detection and response systems more patient centered and how one can enhance their ability to respect patient preferences. Developing systems to improve clinical management is laudable, but somehow we need to also find ways to have these systems make a better connection to what patients want most and what matters most to them, something that may need to include new ways that sometimes suspend use of these systems. At the end of the day, after early detection, patients must have a care experience that they see as an unequivocal improvement.

Acknowledgements

The authors thank our 2 foundation program officers, Dr. Marybeth Sharpe and Ms. Kate Weiland, for their administrative support and encouragement. The authors also thank Dr. Tracy Lieu, Dr. Michelle Caughey, Dr. Philip Madvig, and Ms. Barbara Crawford for their administrative assistance, Dr. Vincent Liu for comments on the manuscript, and Ms. Rachel Lesser for her help with formatting the manuscript and figures.

Disclosures

This work was supported by the Gordon and Betty Moore Foundation, The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. As part of our agreement with the Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Gordon and Betty Moore Foundation and its staff played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. None of the authors has any conflicts of interest to declare of relevance to this work.

Files

References

Hall MJ, Williams SN, DeFrances CJ, Golosinskiy A. Inpatient care for septicemia or sepsis: a challenge for patients and hospitals. NCHS Data Brief. 2011(62):1–8.
Levy MM, Rhodes A, Phillips GS, et al. Surviving sepsis campaign: association between performance metrics and outcomes in a 7.5‐year study. Crit Care Med. 2015;43(1):3–12.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Romero‐Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C‐statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care. 2015;19:285.
Lawn ND, Fletcher DD, Henderson RD, Wolter TD, Wijdicks EF. Anticipating mechanical ventilation in Guillain‐Barre syndrome. Arch Neurol. 2001;58(6):893–898.
Kim YS, Escobar GJ, Halpern SD, Greene JD, Kipnis P, Liu V. The natural history of changes in preferences for life‐sustaining treatments and implications for inpatient mortality in younger and older hospitalized adults. J Am Geriatr Soc. 2016;64(5):981–989.
Sargious A, Lee SJ. Remote collection of questionnaires. Clin Exp Rheumatol. 2014;32(5 suppl 85):S168–S172.
Be prepared to make your health care wishes known. Health care directives. Allina Health website. Available at: http://www.allinahealth.org/Customer-Service/Be-prepared/Be-prepared-to-make-your-health-care-wishes-known. Accessed January 1, 2015.
Patient Reported Outcomes Measurement Information System. Dynamic tools to measure health outcomes from the patient perspective. Available at: http://www.nihpromis.org. Accessed January 15, 2015.
Schorr C, Cinel I, Townsend S, Ramsay G, Levy M, Dellinger RP. Methodology of the surviving sepsis campaign global initiative for improving care of the patient with severe sepsis. Minerva Anestesiol. 2009;75(suppl 1):23–27.
Marshall JC, Dellinger RP, Levy M. The Surviving Sepsis Campaign: a history and a perspective. Surg Infect (Larchmt). 2010;11(3):275–281.
Schorr CA, Dellinger RP. The Surviving Sepsis Campaign: past, present and future. Trends Mol Med. 2014;20(4):192–194.
Shapiro NI, Howell MD, Talmor D, et al. Serum lactate as a predictor of mortality in emergency department patients with infection. Ann Emerg Med. 2005;45(5):524–528.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a Children's Hospital. JAMA. 2007;298(19):2267–2274.
Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ. 2002;324(7334):387–390.
Leach LS, Mayo AM. Rapid response teams: qualitative analysis of their effectiveness. Am J Crit Care. 2013;22(3):198–210.
Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Schorr et al. J Hosp Med. 2016;11:000–000.
Escobar GJ, Turk BJ, Ragins A, et al. Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Liu et al. Liu VX, Morehouse JW, Baker JM, Greene JD, Kipnis P, Gabriel J. Escobar GJ. Data that drive: closing the loop in the learning hospital system. J Hosp Med. 2016;11:000–000.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S5-S10

Read more about Critical Illness Outside the ICU

Sections

Author(s)

R. Phillip Dellinger, MD

Author(s)

R. Phillip Dellinger, MD

Files

Files

Article PDF

Article PDF

CONCEPTUAL FRAMEWORK

JUST WHAT CONSTITUTES EARLY DETECTION (AND HOW DO WE QUANTIFY IT)?

ACTIVATING A RESPONSE ARM

QUANTIFYING BENEFIT

POTENTIAL HARMS OF EARLY DETECTION

ARTICLES IN THIS ISSUE

FUTURE DIRECTIONS: KEY GAPS IN THE FIELD

Acknowledgements

Disclosures

CONCEPTUAL FRAMEWORK

JUST WHAT CONSTITUTES EARLY DETECTION (AND HOW DO WE QUANTIFY IT)?

ACTIVATING A RESPONSE ARM

QUANTIFYING BENEFIT

POTENTIAL HARMS OF EARLY DETECTION

ARTICLES IN THIS ISSUE

FUTURE DIRECTIONS: KEY GAPS IN THE FIELD

Acknowledgements

Disclosures

References

Hall MJ, Williams SN, DeFrances CJ, Golosinskiy A. Inpatient care for septicemia or sepsis: a challenge for patients and hospitals. NCHS Data Brief. 2011(62):1–8.
Levy MM, Rhodes A, Phillips GS, et al. Surviving sepsis campaign: association between performance metrics and outcomes in a 7.5‐year study. Crit Care Med. 2015;43(1):3–12.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Romero‐Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C‐statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care. 2015;19:285.
Lawn ND, Fletcher DD, Henderson RD, Wolter TD, Wijdicks EF. Anticipating mechanical ventilation in Guillain‐Barre syndrome. Arch Neurol. 2001;58(6):893–898.
Kim YS, Escobar GJ, Halpern SD, Greene JD, Kipnis P, Liu V. The natural history of changes in preferences for life‐sustaining treatments and implications for inpatient mortality in younger and older hospitalized adults. J Am Geriatr Soc. 2016;64(5):981–989.
Sargious A, Lee SJ. Remote collection of questionnaires. Clin Exp Rheumatol. 2014;32(5 suppl 85):S168–S172.
Be prepared to make your health care wishes known. Health care directives. Allina Health website. Available at: http://www.allinahealth.org/Customer-Service/Be-prepared/Be-prepared-to-make-your-health-care-wishes-known. Accessed January 1, 2015.
Patient Reported Outcomes Measurement Information System. Dynamic tools to measure health outcomes from the patient perspective. Available at: http://www.nihpromis.org. Accessed January 15, 2015.
Schorr C, Cinel I, Townsend S, Ramsay G, Levy M, Dellinger RP. Methodology of the surviving sepsis campaign global initiative for improving care of the patient with severe sepsis. Minerva Anestesiol. 2009;75(suppl 1):23–27.
Marshall JC, Dellinger RP, Levy M. The Surviving Sepsis Campaign: a history and a perspective. Surg Infect (Larchmt). 2010;11(3):275–281.
Schorr CA, Dellinger RP. The Surviving Sepsis Campaign: past, present and future. Trends Mol Med. 2014;20(4):192–194.
Shapiro NI, Howell MD, Talmor D, et al. Serum lactate as a predictor of mortality in emergency department patients with infection. Ann Emerg Med. 2005;45(5):524–528.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a Children's Hospital. JAMA. 2007;298(19):2267–2274.
Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ. 2002;324(7334):387–390.
Leach LS, Mayo AM. Rapid response teams: qualitative analysis of their effectiveness. Am J Crit Care. 2013;22(3):198–210.
Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Schorr et al. J Hosp Med. 2016;11:000–000.
Escobar GJ, Turk BJ, Ragins A, et al. Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Liu et al. Liu VX, Morehouse JW, Baker JM, Greene JD, Kipnis P, Gabriel J. Escobar GJ. Data that drive: closing the loop in the learning hospital system. J Hosp Med. 2016;11:000–000.

References

Hall MJ, Williams SN, DeFrances CJ, Golosinskiy A. Inpatient care for septicemia or sepsis: a challenge for patients and hospitals. NCHS Data Brief. 2011(62):1–8.
Levy MM, Rhodes A, Phillips GS, et al. Surviving sepsis campaign: association between performance metrics and outcomes in a 7.5‐year study. Crit Care Med. 2015;43(1):3–12.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Vazquez R, Gheorghe C, Grigoriyan A, Palvinskaya T, Amoateng‐Adjepong Y, Manthous CA. Enhanced end‐of‐life care associated with deploying a rapid response team: a pilot study. J Hosp Med. 2009;4(7):449–452.
Smith RL, Hayashi VN, Lee YI, Navarro‐Mariazeta L, Felner K. The medical emergency team call: a sentinel event that triggers goals of care discussion. Crit Care Med. 2014;42(2):322–327.
Romero‐Brufau S, Huddleston JM, Escobar GJ, Liebow M. Why the C‐statistic is not informative to evaluate early warning scores and what metrics to use. Crit Care. 2015;19:285.
Lawn ND, Fletcher DD, Henderson RD, Wolter TD, Wijdicks EF. Anticipating mechanical ventilation in Guillain‐Barre syndrome. Arch Neurol. 2001;58(6):893–898.
Kim YS, Escobar GJ, Halpern SD, Greene JD, Kipnis P, Liu V. The natural history of changes in preferences for life‐sustaining treatments and implications for inpatient mortality in younger and older hospitalized adults. J Am Geriatr Soc. 2016;64(5):981–989.
Sargious A, Lee SJ. Remote collection of questionnaires. Clin Exp Rheumatol. 2014;32(5 suppl 85):S168–S172.
Be prepared to make your health care wishes known. Health care directives. Allina Health website. Available at: http://www.allinahealth.org/Customer-Service/Be-prepared/Be-prepared-to-make-your-health-care-wishes-known. Accessed January 1, 2015.
Patient Reported Outcomes Measurement Information System. Dynamic tools to measure health outcomes from the patient perspective. Available at: http://www.nihpromis.org. Accessed January 15, 2015.
Schorr C, Cinel I, Townsend S, Ramsay G, Levy M, Dellinger RP. Methodology of the surviving sepsis campaign global initiative for improving care of the patient with severe sepsis. Minerva Anestesiol. 2009;75(suppl 1):23–27.
Marshall JC, Dellinger RP, Levy M. The Surviving Sepsis Campaign: a history and a perspective. Surg Infect (Larchmt). 2010;11(3):275–281.
Schorr CA, Dellinger RP. The Surviving Sepsis Campaign: past, present and future. Trends Mol Med. 2014;20(4):192–194.
Shapiro NI, Howell MD, Talmor D, et al. Serum lactate as a predictor of mortality in emergency department patients with infection. Ann Emerg Med. 2005;45(5):524–528.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a Children's Hospital. JAMA. 2007;298(19):2267–2274.
Buist MD, Moore GE, Bernard SA, Waxman BP, Anderson JN, Nguyen TV. Effects of a medical emergency team on reduction of incidence of and mortality from unexpected cardiac arrests in hospital: preliminary study. BMJ. 2002;324(7334):387–390.
Leach LS, Mayo AM. Rapid response teams: qualitative analysis of their effectiveness. Am J Crit Care. 2013;22(3):198–210.
Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Schorr et al. J Hosp Med. 2016;11:000–000.
Escobar GJ, Turk BJ, Ragins A, et al. Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Liu et al. Liu VX, Morehouse JW, Baker JM, Greene JD, Kipnis P, Gabriel J. Escobar GJ. Data that drive: closing the loop in the learning hospital system. J Hosp Med. 2016;11:000–000.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S5-S10

Page Number

S5-S10

Publications

Publications

Article Type

Display Headline

Early detection, prevention, and mitigation of critical illness outside intensive care settings

Display Headline

Early detection, prevention, and mitigation of critical illness outside intensive care settings

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Gabriel J. Escobar, MD, Systems Research Initiative, Kaiser Permanente Division of Research, Kaiser Permanente, 2000 Broadway Avenue, 032 R01, Oakland, CA 94612‐2304; Telephone: 510‐891‐5929; Fax: 510‐891‐3606; E‐mail: gabriel.escobar@kp.org

Content Gating

Gated (full article locked unless allowed per User)

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

Media Files

Incorporating an EWS Into Practice

Article Type

Changed

Mon, 01/02/2017 - 19:34

Display Headline

Incorporating an Early Detection System Into Routine Clinical Practice in Two Community Hospitals

Author(s)

Elizabeth Scruth, RN, MPH, PhD

Carmen Adams, DNSc, RNC

Margaret Guo, MD, PhD

Patients who deteriorate outside highly monitored settings and who require unplanned transfer to the intensive care unit (ICU) are known to have high mortality and morbidity.[1, 2, 3, 4, 5] The notion that early detection of a deteriorating patient improves outcomes has intuitive appeal and is discussed in a large number of publications.[6, 7, 8, 9, 10] However, much less information is available on what should be done after early detection is made.[11] Existing literature on early warning systems (EWSs) does not provide enough detail to serve as a map for implementation. This lack of transparency is complicated by the fact that, although the comprehensive inpatient electronic medical record (EMR) now constitutes the central locus for clinical practice, much of the existing literature comes from research institutions that may employ home‐grown EMRs, not community hospitals that employ commercially available systems.

In this issue of the Journal of Hospital Medicine, we describe our efforts to bridge that gap by implementing an EWS in a pair of community hospitals. The EWS's development and its basic statistical and electronic infrastructure are described in the articles by Escobar and Dellinger and Escobar et al.[2, 12, 13] In this report, we focus on how we addressed clinicians' primary concern: What do we do when we get an alert? Because it is described in detail by Granich et al.[14] elsewhere in this issue of the Journal of Hospital Medicine, a critical component of our implementation process (ensuring that patient preferences with respect to supportive care are honored) is not discussed.

Our article is divided into the following sections: rationale, preimplementation preparatory work, workflow development, response protocols, challenges and key learnings, and concluding reflections.

RATIONALE

Much of the previous work on the implementation of alarm systems has focused on the statistics behind detection or on the quantification of processes (eg, how many rapid response calls were triggered) or on outcomes such as mortality. The conceptual underpinnings and practical steps necessary for successful integration of an alarm system into the clinicians' workflow have not been articulated. Our theoretical framework was based on (1) improving situational awareness[15] (knowing what is going on around you and what is likely to happen next) and (2) mitigating cognitive errors.

An EWS enhances situational awareness most directly by earlier identification of a problem with a particular patient. As is detailed by Escobar et al.[16] in this issue of the Journal of Hospital Medicine, our EWS extracts EMR data every 6 hours, performs multiple calculations, and then displays 3 scores in real time in the inpatient dashboard (known as the Patient Lists activity in the Epic EMR). The first of these scores is the Laboratory‐Based Acute Physiologic Score, version 2 (LAPS2), an objective severity score whose retrospective version is already in use in Kaiser Permanente Northern California (KPNC) for internal benchmarking.[13] This score captures a patient's overall degree of physiologic instability within the preceding 72 hours. The second is the Comorbidity Point Score, version 2 (COPS2), a longitudinal comorbidity score based on the patient's diagnoses over the preceding 12 months.[13] This score captures a patient's overall comorbidity burden. Thus, it is possible for a patient to be very ill (high COPS2) while also being stable (low LAPS2) or vice versa. Both of these scores have other uses, including prediction of rehospitalization risk in real time,[17] which is also being piloted at KPNC. Finally, the Advanced Alert Monitoring (AAM) score, which integrates the LAPS2 and COPS2 with other variables, provides a 12‐hour deterioration risk, with a threshold value of 8% triggering response protocols. At or above this threshold, which was agreed to prior to implementation, the system achieves 25% sensitivity, 98% specificity, with a number needed to evaluate of 10 to 12, a level of workload that was felt to be acceptable by clinicians. Actions triggered by the EWS may be quite different from those one would take when being notified of a code blue, which is called at the time an event occurs. The EWS focuses attention on patients who might be missed because they do not yet appear critically ill. It also provides a shared, quantifiable measure of a patient's risk that can trigger a standardized plan of action to follow in evaluating and treating a patient.[15]

In addition to enhancing situational awareness, we intended the alarms to produce cognitive change in practitioners. Our goal was to replace medical intuition with analytic, evidence‐based judgment of future illness. We proceeded with the understanding that replacing quick intuition with slower analytic response is an essential skill in developing sound clinical reasoning.[18, 19, 20] The alert encourages physicians to reassess high‐risk patients facilitating a cognitive shift from automatic, error‐prone processing to slower, deliberate processing. Given the busy pace of ward work, slowing down permits clinicians to reassess previously overlooked details. Related to this process of inducing cognitive change is a secondary effect: we uncovered and discussed physician biases. Physicians are subject to potential biases that allow patients to deteriorate.[18, 19, 20] Therefore, we addressed bias through education. By reviewing particular cases of unanticipated deterioration at each hospital facility, we provided evidence for the problem of in‐hospital deterioration. This framed the new tool as an opportunity for improving treatment and encouraged physicians to act on the alert using a structured process.

INTERVENTIONS

Preimplementation Preparatory Work

Initial KPNC data provided strong support for the generally accepted notion that unplanned transfer patients have poor outcomes.[2, 4, 5] However, published reports failed to provide the granular detail clinicians need to implement a response arm at the unit and patient level. In preparation for going live, we conducted a retrospective chart review. This included data from patients hospitalized from January 1, 2011 through December 31, 2012 (additional detail is provided in the Supporting Information, Appendix, in the online version of this article). The key findings from our internal review of subjective documentation preceding deterioration are similar to those described in the literature and summarized in Figure 1, which displays the 5 most common clinical presentations associated with unplanned transfers.

Results of and internal chart review summary of the most common clinical presentations among patients who experienced unplanned transfer to the intensive care unit (left panel) or who died on the ward or transitional care unit with a full code care directive. Numbers do not add up to 100% because some patients had more than 1 problem. See text and online appendix for additional details.

The chart review served several major roles. First, it facilitated cognitive change by eliminating the notion that it can't happen here. Second, it provided considerable guidance on key clinical components that had to be incorporated into the workflow. Third, it engaged the rapid response team (RRT) in reviewing our work retrospectively to identify future opportunities. Finally, the review provided considerable guidance with respect to structuring documentation requirements.

As a result of the above efforts, other processes detailed below, and knowledge described in several of the companion articles in this issue of the Journal of Hospital Medicine, 3 critical elements, which had been explicitly required by our leadership, were in place prior to the go‐live date: a general consensus among hospitalists and nurses that this would be worth testing, a basic clinical response workflow, and an automated checklist for documentation. We refined these in a 2‐week shadowing phase preceding the start date. In this phase, the alerts were not displayed in the EMR. Instead, programmers working on the project notified selected physician leaders by phone. This permitted them to understand exactly what sort of patients were reaching the physiologic threshold so that they could better prepare both RRT registered nurses (RNs) and hospitalists for the go‐live date. This also provided an opportunity to begin refining the documentation process using actual patients.

The original name for our project was Early Detection of Impending Physiologic Deterioration. However, during the preparatory phase, consultation with our public relations staff led to a concern that the name could be frightening to some patients. This highlights the need to consider patient perceptions and how words used in 1 way by physicians can have different connotations to nonclinicians. Consequently, the system was renamed, and it is now referred to as Advance Alert Monitoring (AAM).

Workflow Development

We carefully examined the space where electronic data, graphical user interfaces, and clinical practice blend, a nexus now commonly referred to as workflow or user experience.[21] To promote situational awareness and effect cognitive change, we utilized the Institute for Health Care Improvement's Plan‐Do‐Study‐Act model.[22, 23] We then facilitated the iterative development of a clinician‐endorsed workflow.[22, 23, 24, 25] By adjusting the workflow based on ongoing experience and giving clinicians multiple opportunities to revise (a process that continues to date), we ensured clinicians would approach and endorse the alarm system as a useful tool for decision support.

Table 1 summarizes the work groups assembled for our implementation, and Table 2 provides a system‐oriented checklist indicating key components that need to be in place prior to having an early warning system go live in a hospital. Figure 2 summarizes the alert response protocols we developed through an iterative process at the 2 pilot sites. The care path shown in Figure 2 is the result of considerable revision, mostly due to actual experience acquired following the go live date. The diagram also includes a component that is still work in progress. This is how an emergency department probability estimate (triage support) will be integrated into both the ward as well as the ICU workflows. Although this is beyond the scope of this article, other hospitals may be experimenting with triage support (eg, for sepsis patients), so it is important to consider how one would incorporate such support into workflows.

Table 1.Workgroups Established for Early Warning System Rollout
Workgroup	Goals
NOTE: Abbreviations: POLST, physician orders for life‐sustaining treatment.
Clinical checklist	Perform structured chart review of selected unplanned transfer patients and near misses
Clinical checklist	Develop a checklist for mitigation strategies given an alert
Develop documentation standards given an alert
Develop escalation protocol given an alert
Workload and threshold	Determine threshold for sensitivity of alerts and resulting impact on clinician workload
Patient preferences	Prepare background information to be presented to providers regarding end‐of‐life care and POLST orders
Patient preferences	Coordinate with clinical checklist workgroup to generate documentation templates that provide guidance for appropriate management of patients regarding preferences on escalation of care and end‐of‐life care
Electronic medical record coordination	Review proposed electronic medical record changes
Electronic medical record coordination	Make recommendation for further changes as needed
Develop plan for rollout of new and/or revised electronic record tools
Designate contact list for questions/emssues that may arise regarding electronic record changes during the pilot
Determine alert display choices and mode of alert notification
Nursing committee	Review staffing needs in anticipation of alert
Nursing committee	Coordinate with workload and threshold group
Develop training calendar to ensure skills necessary for successful implementation of alerts
Make recommendations for potential modification of rapid response team's role in development of a clinical checklist for nurses responding to an alert
Design educational materials for clinicians
Local communication strategy	Develop internal communication plan (for clinical staff not directly involved with pilot)
	Develop external communication plan (for nonclinicians who may hear about the project)

Table 2.Hospital System‐Wide Go Live Checklist
Level	Tasks
Administration	Obtain executive committee approval
Administration	Establish communication protocols with quality assurance and quality improvement committees
Review protocols with medicallegal department
Communication	Write media material for patients and families
Communication	Develop and disseminate scripts for front‐line staff
Develop communication and meet with all relevant front‐line staff on merits of project
Educate all staff on workflow changes and impacts
Clinical preparation	Conduct internal review of unplanned transfers and present results to all clinicians
Clinical preparation	Determine service level agreements, ownership of at‐risk patients, who will access alerts
Conduct staff meetings to educate staff
Perform debriefs on relevant cases
Determine desired outcomes, process measures, balancing measures
Determine acceptable clinician burden (alerts/day)
Technology	Establish documentation templates
Technology	Ensure access to new data fields (electronic medical record security process must be followed for access rights)
Workflows	Workflows (clinical response, patient preferences, supportive care, communication, documentation) must be in place prior to actual go live
Shadowing	Testing period (alerts communicated to selected clinicians prior to going live) should occur

Clinical response workflow at pilot sites integration of clinical teams with automated deterioration probability estimates generated every 6 hours. Note that, because they are calibrated to 12‐hour lead time, AAM alerts are given third priority (code blue gets first priority, regular RRT call gets second priority). *Where the SSF and SAC workflows are different. Abbreviations: AAM, advance alert monitor; ATN, action team nurse; COPS, Comorbidity Point Score; ED, emergency department; EHR, electronic health record; EMR, electronic medical record; HC, Health Connect, Kaiser Permanente implementation of EPIC Electronic Health Record; HBS, hospitalist; ICU, intensive care unit; LAPS, Laboratory‐Based Acute Physiology Score; LCP, life care plan (patient preferences regarding life sustaining treatments); MD, medical doctor; MSW, medical social worker; PC, palliative care; RN, registered nurse; RRT, rapid response nurse; SAC, Sacramento Kaiser; SCT, supportive care team (includes palliative care); SSF, South San Francisco; SW, social worker.

RESPONSE PROTOCOLS

At South San Francisco, the RRT consists of an ICU nurse, a respiratory care therapist, and a designated hospitalist; at Sacramento, the team is also augmented by an additional nurse (the house supervisor). In addition to responding to the AAM alerts, RRT nurses respond to other emergency calls such as code blues, stroke alerts, and patient or patient‐familyinitiated rapid response calls. They also expedite time sensitive workups and treatments. They check up on recent transfers from the ICU to ensure continued improvement justifying staying on the ward. Serving as peer educators, they assist with processes such as chest tube or central line insertions, troubleshoot high‐risk medication administration, and ensure that treatment bundles (eg, for sepsis) occur expeditiously.

The RRT reviews EWS scores every 6 hours. The AAM score is seen as soon as providers open the chart, which helps triage patients for evaluation. Because patients can still be at risk even without an elevated AAM score, all normal escalation pathways remain in place. Once an alert is noted in the inpatient dashboard, the RRT nurse obtains a fresh set of vital signs, assesses the patient's clinical status, and informs the physician, social worker, and primary nurse (Figure 2). Team members work with the bedside nurse, providing support with assessment, interventions, plans, and follow‐up. Once advised of the alert, the hospitalist performs a second chart review and evaluates the patient at the bedside to identify factors that could underlie potential deterioration. After this evaluation, the hospitalist documents concerns, orders appropriate interventions (which can include escalation), and determines appropriate follow‐up. We made sure the team knew that respiratory distress, arrhythmias, mental status changes, or worsening infection were responsible for over 80% of in‐hospital deterioration cases. We also involved palliative care earlier in patient care, streamlining the process so the RRT makes just 1 phone call to the social worker, who contacts the palliative care physician and nurse to ensure patients have a designated surrogate in the event of further deterioration.

Our initial documentation template consisted of a comprehensive organ system‐based physician checklist. However, although this was of use to covering physicians unfamiliar with a given patient, it was redundant and annoying to attending providers already familiar with the patient. After more than 30 iterations, we settled on a succinct note that only documented the clinicians' clinical judgment as to what constituted the major risk for deterioration and what the mitigation strategies would be. Both of these judgments are in a checklist format (see Supporting Information, Appendix, in the online version of this article for the components of the physician and nurse notes).

Prior to the implementation of the system, RRT nurses performed proactive rounding by manually checking patient labs and vital signs, an inefficient process due to the poor sensitivity and specificity of individual values. Following implementation of the system, RRT RNs and clinicians switched to sorting patients by the 3 scores (COPS2, LAPS2, AAM). For example, patients may be stable at admission (as evidenced by their AAM score) but be at high risk due to their comorbidities. One approach that has been employed is to proactively check such patients to ensure they have a care directive in place, as is described in the article by Granich et al.[14] The Supportive Care Team (detailed in Granich et al.) assesses needs for palliative care and provides in‐hospital consultation as needed. Social services staff perform chart reviews to ensure a patient surrogate has been defined and also works with patients and their families to clarify goals of care.

CHALLENGES AND KEY LEARNINGS

One challenge that arose was reconciling the periodic nature of the alert (every 6 hours) with physicians' availability, which varied due to different rounding workflows at the 2 sites. Consequently, the alert cycle was changed; at the first site, the cycle was set to 1000‐1600‐2200‐0400, whereas the second site chose 0800‐1400‐2000‐0200.

One essential but problematic component of the clinical response is the issue of documentation. Inadequate documentation could lead to adverse outcomes, clinician malpractice exposure, and placing the entire hospital at risk for enterprise liability when clinical responses are not documented. This issue is complicated by the fact that overzealous efforts could lead to less or no documentation by making it too onerous for busy clinicians. We found that the ease with which data can populate progress notes in the EMR can lead to note bloat. Clearly, no documentation is not enough, and a complete history and physical is too much. Paradoxically, 1 of the issues underlying our problems with documentation was the proactive nature of the alerts themselves; because they are based on an outcome prediction in the next 12 hours, documenting the response to them may lack (perceived) urgency.

Shortly after the system went live, a patient who had been recently transferred out to the ward from the ICU triggered an alert. As a response was mounted, the team realized that existing ward protocols did not specify which physician service (intensivist or hospitalist) was responsible for patients who were transitioning from 1 unit to another. We also had to perform multiple revisions of the protocols specifying how alerts were handled when they occurred at times of change of shift. Eventually, we settled on having the combination of a hospitalist and an RRT nurse as the cornerstone of the response, with the hospitalist service as the primary owner of the entire process, but this arrangement might need to be varied in different settings. As a result of the experience with the pilot, the business case for deployment in the remaining 19 hospitals includes a formal budget request so that all have properly staffed RRTs, although the issue of primary ownership of the alert process for different patient types (eg, surgical patients) will be decided on a hospital‐by‐hospital basis. These experiences raise the intriguing possibility that implementation of alert systems can lead to the identification of systemic gaps in existing protocols. These gaps can include specific components of the hospital service agreements between multiple departments (emergency, hospital medicine, ICU, palliative care, surgery) as well as problems with existing workflows.

In addition to ongoing tweaking of care protocols, 3 issues remain unresolved. First is the issue of documentation. The current documentation notes are not completely satisfactory, and we are working with the KPNC EMR administrators to refine the tool. Desirable refinements include (1) having the system scores populate in more accessible sectors of the EMR where their retrieval will facilitate increased automation of the note writing process, (2) changing the note type to a note that will facilitate process audits, and (3) linking the note to other EMR tools so that the response arm can be tracked more formally. The second issue is the need to develop strategies to address staff turnover; for example, newer staff may not have received the same degree of exposure to the system as those who were there when it was started. Finally, due to limited resources, we have done very limited work on more mechanistic analyses of the clinical response itself. For example, it would be desirable to perform a formal quantitative, risk‐adjusted process‐outcome analysis of why some patients' outcomes are better than others following an alert.

Finally, it is also the case that we have had some unexpected occurrences that hint at new uses and benefits of alert systems. One of these is the phenomenon of chasing the alert. Some clinicians, on their own, have taken a more proactive stance in the care of patients in whom the AAM score is rising or near the alert threshold. This has 2 potential consequences. Some patients are stabilized and thus do not reach threshold instability levels. In other cases, patients reach threshold but the response team is informed that things are already under control. A second unexpected result is increased requests for COPS2 scores by clinicians who have heard about the system, particularly surgeons who would like to use the comorbidity scores as a screening tool in the outpatient setting. Because KPNC is an integrated system, it is not likely that such alternatives will be implemented immediately without considerable analysis, but it is clear that the system's deployment has captured the clinicians' imagination.

CONCLUSIONS AND FUTURE DIRECTIONS

Our preparatory efforts have been successful. We have found that embedding an EWS in a commercially available EMR is acceptable to hospital physicians and nurses. We have developed a coordinated workflow for mitigation and escalation that is tightly linked to the availability of probabilistic alerts in real time. Although resource limitations have precluded us from conducting formal clinician surveys, the EWS has been discussed at multiple hospital‐wide as well as department‐specific meetings. Although there have been requests for clarification, refinements, and modifications in workflows, no one has suggested that the system be discontinued. Further, many of the other KPNC hospitals have requested that the EWS be deployed at their site. We have examined KPNC databases that track patient complaints and have not found any complaints that could be linked to the EWS. Most importantly, the existence of the workflows we have developed has played a major role in KPNC's decision to deploy the system in its remaining hospitals.

Although alert fatigue is the number 1 reason that clinicians do not utilize embedded clinical decision support,[26] simply calibrating statistical models is insufficient. Careful consideration of clinicians' needs and responsibilities, particularly around ownership of patients and documentation, is essential. Such consideration needs to include planning time and socializing the system (providing multiple venues for clinicians to learn about the system as well as participate in the process for using it).

We anticipate that, as the system leaves the pilot stage and becomes a routine component of hospital care, additional enhancements (eg, sending notifications to smart phones, providing an alert response tracking system) will be added. Our organization is also implementing real‐time concurrent review of inpatient EMRs (eg, for proactive detection of an expanded range of potential process failures), and work is underway on how to link the workflows we describe here with this effort. As has been the case with other systems,[27] it is likely that we will eventually move to continuous scanning of patient data rather than only every 6 hours. Given that the basic workflow is quite robust and amenable to local modifications, we are confident that our clinicians and hospitals will adapt to future system enhancements.

Lastly, we intend to conduct additional research on the clinical response itself. In particular, we consider it extremely important to conduct formal quantitative analyses on why some patients' outcomes are better than others following an alert. A key component of this effort will be to develop tools that can permit an automatedor nearly automatedassessment of the clinical response. For example, we are considering automated approaches that would scan the EMR for the presence of specific orders, notes, vital signs patterns, and laboratory tests following an alert. Whereas it may not be possible to dispense with manual chart review, even partial automation of a feedback process could lead to significant enhancement of our quality improvement efforts.

Acknowledgements

The authors thank Dr. Michelle Caughey, Dr. Philip Madvig, Dr. Brian Hoberman, Dr. Patricia Conolly, and Ms. Barbara Crawford for their administrative support; Dr. Tracy Lieu for reviewing the manuscript; and Ms. Rachel Lesser for formatting the manuscript. The authors also thank Drs. Jason Anderson, John Fitzgibbon, Elena M. Nishimura, and Najm Haq for their support of the project. We are particularly grateful to our nurses, Theresa A. Villorente, Zoe Sutton, Doanh Ly, Catherine Burger, and Hillary R. Mitchell, for their critical assistance. Last but not least, we also thank all the hospitalists and nurses at the Kaiser Permanente Sacramento and South San Francisco hospitals.

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. As part of our agreement with the Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Gordon and Betty Moore Foundation played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component; the same was the case with the other sponsors. None of the authors has any conflicts of interest to declare of relevance to this work

Files

References

Gerber DR, Schorr C, Ahmed I, Dellinger RP, Parrillo J. Location of patients before transfer to a tertiary care intensive care unit: impact on outcome. J Crit Care. 2009;24(1):108–113.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8(1):13–19.
Mailey J, Digiovine B, Baillod D, Gnam G, Jordan J, Rubinfeld I. Reducing hospital standardized mortality rate with early interventions. J Trauma Nursing. 2006;13(4):178–182.
Tarassenko L, Clifton DA, Pinsky MR, Hravnak MT, Woods JR, Watkinson PJ. Centile‐based early warning scores derived from statistical distributions of vital signs. Resuscitation. 2011;82(8):1013–1018.
Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med. 2012;40(7):2096–2101.
Zimlichman E, Szyper‐Kravitz M, Shinar Z, et al. Early recognition of acutely deteriorating patients in non‐intensive care units: assessment of an innovative monitoring technology. J Hosp Med. 2012;7(8):628–633.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Brady PW, Muething S, Kotagal U, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics. 2013;131(1):e298–e308.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Granich R, Sutton Z, Kim Y. et al. Early detection of critical illness outside the intensive care unit: clarifying treatment plans and honoring goals of care using a supportive care team. J Hosp Med. 2016;11:000–000.
Brady PW, Goldenhar LM. A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk. BMJ Qual Saf. 2014;23(2):153–161.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record‐based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and postdischarge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78(8):775–780.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf. 2013;22(suppl 2):ii58–ii64.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf. 2013;22(suppl 2):ii65–ii72.
El‐Kareh R, Hasan O, Schiff GD. Use of health information technology to reduce diagnostic errors. BMJ Qual Saf. 2013;22(suppl 2):ii40–ii51.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002;77(10):981–992.
Goldenhar LM, Brady PW, Sutcliffe KM, Muething SE. Huddling for high reliability and situation awareness. BMJ Qual Saf. 2013;22(11):899–906.
Top 10 patient safety concerns for healthcare organizations. ECRI Institute website. Available at: https://www.ecri.org/Pages/Top‐10‐Patient‐Safety‐Concerns.aspx. Accessed February 18, 2016.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S25-S31

Read more about Incorporating an EWS Into Practice

Sections

Author(s)

Elizabeth Scruth, RN, MPH, PhD

Carmen Adams, DNSc, RNC

Margaret Guo, MD, PhD

Author(s)

Elizabeth Scruth, RN, MPH, PhD

Carmen Adams, DNSc, RNC

Margaret Guo, MD, PhD

Files

Files

Article PDF

Article PDF

Our article is divided into the following sections: rationale, preimplementation preparatory work, workflow development, response protocols, challenges and key learnings, and concluding reflections.

RATIONALE

INTERVENTIONS

Preimplementation Preparatory Work

Workflow Development

Table 1.Workgroups Established for Early Warning System Rollout
Workgroup	Goals
NOTE: Abbreviations: POLST, physician orders for life‐sustaining treatment.
Clinical checklist	Perform structured chart review of selected unplanned transfer patients and near misses
Clinical checklist	Develop a checklist for mitigation strategies given an alert
Develop documentation standards given an alert
Develop escalation protocol given an alert
Workload and threshold	Determine threshold for sensitivity of alerts and resulting impact on clinician workload
Patient preferences	Prepare background information to be presented to providers regarding end‐of‐life care and POLST orders
Patient preferences	Coordinate with clinical checklist workgroup to generate documentation templates that provide guidance for appropriate management of patients regarding preferences on escalation of care and end‐of‐life care
Electronic medical record coordination	Review proposed electronic medical record changes
Electronic medical record coordination	Make recommendation for further changes as needed
Develop plan for rollout of new and/or revised electronic record tools
Designate contact list for questions/emssues that may arise regarding electronic record changes during the pilot
Determine alert display choices and mode of alert notification
Nursing committee	Review staffing needs in anticipation of alert
Nursing committee	Coordinate with workload and threshold group
Develop training calendar to ensure skills necessary for successful implementation of alerts
Make recommendations for potential modification of rapid response team's role in development of a clinical checklist for nurses responding to an alert
Design educational materials for clinicians
Local communication strategy	Develop internal communication plan (for clinical staff not directly involved with pilot)
	Develop external communication plan (for nonclinicians who may hear about the project)

Table 2.Hospital System‐Wide Go Live Checklist
Level	Tasks
Administration	Obtain executive committee approval
Administration	Establish communication protocols with quality assurance and quality improvement committees
Review protocols with medicallegal department
Communication	Write media material for patients and families
Communication	Develop and disseminate scripts for front‐line staff
Develop communication and meet with all relevant front‐line staff on merits of project
Educate all staff on workflow changes and impacts
Clinical preparation	Conduct internal review of unplanned transfers and present results to all clinicians
Clinical preparation	Determine service level agreements, ownership of at‐risk patients, who will access alerts
Conduct staff meetings to educate staff
Perform debriefs on relevant cases
Determine desired outcomes, process measures, balancing measures
Determine acceptable clinician burden (alerts/day)
Technology	Establish documentation templates
Technology	Ensure access to new data fields (electronic medical record security process must be followed for access rights)
Workflows	Workflows (clinical response, patient preferences, supportive care, communication, documentation) must be in place prior to actual go live
Shadowing	Testing period (alerts communicated to selected clinicians prior to going live) should occur

RESPONSE PROTOCOLS

CHALLENGES AND KEY LEARNINGS

CONCLUSIONS AND FUTURE DIRECTIONS

Acknowledgements

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. As part of our agreement with the Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Gordon and Betty Moore Foundation played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component; the same was the case with the other sponsors. None of the authors has any conflicts of interest to declare of relevance to this work

Our article is divided into the following sections: rationale, preimplementation preparatory work, workflow development, response protocols, challenges and key learnings, and concluding reflections.

RATIONALE

INTERVENTIONS

Preimplementation Preparatory Work

Workflow Development

Table 1.Workgroups Established for Early Warning System Rollout
Workgroup	Goals
NOTE: Abbreviations: POLST, physician orders for life‐sustaining treatment.
Clinical checklist	Perform structured chart review of selected unplanned transfer patients and near misses
Clinical checklist	Develop a checklist for mitigation strategies given an alert
Develop documentation standards given an alert
Develop escalation protocol given an alert
Workload and threshold	Determine threshold for sensitivity of alerts and resulting impact on clinician workload
Patient preferences	Prepare background information to be presented to providers regarding end‐of‐life care and POLST orders
Patient preferences	Coordinate with clinical checklist workgroup to generate documentation templates that provide guidance for appropriate management of patients regarding preferences on escalation of care and end‐of‐life care
Electronic medical record coordination	Review proposed electronic medical record changes
Electronic medical record coordination	Make recommendation for further changes as needed
Develop plan for rollout of new and/or revised electronic record tools
Designate contact list for questions/emssues that may arise regarding electronic record changes during the pilot
Determine alert display choices and mode of alert notification
Nursing committee	Review staffing needs in anticipation of alert
Nursing committee	Coordinate with workload and threshold group
Develop training calendar to ensure skills necessary for successful implementation of alerts
Make recommendations for potential modification of rapid response team's role in development of a clinical checklist for nurses responding to an alert
Design educational materials for clinicians
Local communication strategy	Develop internal communication plan (for clinical staff not directly involved with pilot)
	Develop external communication plan (for nonclinicians who may hear about the project)

Table 2.Hospital System‐Wide Go Live Checklist
Level	Tasks
Administration	Obtain executive committee approval
Administration	Establish communication protocols with quality assurance and quality improvement committees
Review protocols with medicallegal department
Communication	Write media material for patients and families
Communication	Develop and disseminate scripts for front‐line staff
Develop communication and meet with all relevant front‐line staff on merits of project
Educate all staff on workflow changes and impacts
Clinical preparation	Conduct internal review of unplanned transfers and present results to all clinicians
Clinical preparation	Determine service level agreements, ownership of at‐risk patients, who will access alerts
Conduct staff meetings to educate staff
Perform debriefs on relevant cases
Determine desired outcomes, process measures, balancing measures
Determine acceptable clinician burden (alerts/day)
Technology	Establish documentation templates
Technology	Ensure access to new data fields (electronic medical record security process must be followed for access rights)
Workflows	Workflows (clinical response, patient preferences, supportive care, communication, documentation) must be in place prior to actual go live
Shadowing	Testing period (alerts communicated to selected clinicians prior to going live) should occur

RESPONSE PROTOCOLS

CHALLENGES AND KEY LEARNINGS

CONCLUSIONS AND FUTURE DIRECTIONS

Acknowledgements

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. As part of our agreement with the Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Gordon and Betty Moore Foundation played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component; the same was the case with the other sponsors. None of the authors has any conflicts of interest to declare of relevance to this work

References

Gerber DR, Schorr C, Ahmed I, Dellinger RP, Parrillo J. Location of patients before transfer to a tertiary care intensive care unit: impact on outcome. J Crit Care. 2009;24(1):108–113.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8(1):13–19.
Mailey J, Digiovine B, Baillod D, Gnam G, Jordan J, Rubinfeld I. Reducing hospital standardized mortality rate with early interventions. J Trauma Nursing. 2006;13(4):178–182.
Tarassenko L, Clifton DA, Pinsky MR, Hravnak MT, Woods JR, Watkinson PJ. Centile‐based early warning scores derived from statistical distributions of vital signs. Resuscitation. 2011;82(8):1013–1018.
Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med. 2012;40(7):2096–2101.
Zimlichman E, Szyper‐Kravitz M, Shinar Z, et al. Early recognition of acutely deteriorating patients in non‐intensive care units: assessment of an innovative monitoring technology. J Hosp Med. 2012;7(8):628–633.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Brady PW, Muething S, Kotagal U, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics. 2013;131(1):e298–e308.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Granich R, Sutton Z, Kim Y. et al. Early detection of critical illness outside the intensive care unit: clarifying treatment plans and honoring goals of care using a supportive care team. J Hosp Med. 2016;11:000–000.
Brady PW, Goldenhar LM. A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk. BMJ Qual Saf. 2014;23(2):153–161.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record‐based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and postdischarge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78(8):775–780.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf. 2013;22(suppl 2):ii58–ii64.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf. 2013;22(suppl 2):ii65–ii72.
El‐Kareh R, Hasan O, Schiff GD. Use of health information technology to reduce diagnostic errors. BMJ Qual Saf. 2013;22(suppl 2):ii40–ii51.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002;77(10):981–992.
Goldenhar LM, Brady PW, Sutcliffe KM, Muething SE. Huddling for high reliability and situation awareness. BMJ Qual Saf. 2013;22(11):899–906.
Top 10 patient safety concerns for healthcare organizations. ECRI Institute website. Available at: https://www.ecri.org/Pages/Top‐10‐Patient‐Safety‐Concerns.aspx. Accessed February 18, 2016.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.

References

Gerber DR, Schorr C, Ahmed I, Dellinger RP, Parrillo J. Location of patients before transfer to a tertiary care intensive care unit: impact on outcome. J Crit Care. 2009;24(1):108–113.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2013;8(1):13–19.
Mailey J, Digiovine B, Baillod D, Gnam G, Jordan J, Rubinfeld I. Reducing hospital standardized mortality rate with early interventions. J Trauma Nursing. 2006;13(4):178–182.
Tarassenko L, Clifton DA, Pinsky MR, Hravnak MT, Woods JR, Watkinson PJ. Centile‐based early warning scores derived from statistical distributions of vital signs. Resuscitation. 2011;82(8):1013–1018.
Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med. 2012;40(7):2096–2101.
Zimlichman E, Szyper‐Kravitz M, Shinar Z, et al. Early recognition of acutely deteriorating patients in non‐intensive care units: assessment of an innovative monitoring technology. J Hosp Med. 2012;7(8):628–633.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Brady PW, Muething S, Kotagal U, et al. Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics. 2013;131(1):e298–e308.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Granich R, Sutton Z, Kim Y. et al. Early detection of critical illness outside the intensive care unit: clarifying treatment plans and honoring goals of care using a supportive care team. J Hosp Med. 2016;11:000–000.
Brady PW, Goldenhar LM. A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk. BMJ Qual Saf. 2014;23(2):153–161.
Escobar G, Turk B, Ragins A, et al. Piloting electronic medical record‐based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11:000–000.
Escobar GJ, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and postdischarge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;78(8):775–780.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 1: origins of bias and theory of debiasing. BMJ Qual Saf. 2013;22(suppl 2):ii58–ii64.
Croskerry P, Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf. 2013;22(suppl 2):ii65–ii72.
El‐Kareh R, Hasan O, Schiff GD. Use of health information technology to reduce diagnostic errors. BMJ Qual Saf. 2013;22(suppl 2):ii40–ii51.
Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: Jossey‐Bass; 2009.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Graber M, Gordon R, Franklin N. Reducing diagnostic errors in medicine: what's the goal? Acad Med. 2002;77(10):981–992.
Goldenhar LM, Brady PW, Sutcliffe KM, Muething SE. Huddling for high reliability and situation awareness. BMJ Qual Saf. 2013;22(11):899–906.
Top 10 patient safety concerns for healthcare organizations. ECRI Institute website. Available at: https://www.ecri.org/Pages/Top‐10‐Patient‐Safety‐Concerns.aspx. Accessed February 18, 2016.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S25-S31

Page Number

S25-S31

Publications

Publications

Article Type

Display Headline

Incorporating an Early Detection System Into Routine Clinical Practice in Two Community Hospitals

Display Headline

Incorporating an Early Detection System Into Routine Clinical Practice in Two Community Hospitals

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: B. Alex Dummett, MD, Advance Alert Monitor Clinical Lead, Kaiser Permanente Medical Center, 5th Floor HBS Office, 1200 El Camino Real, South San Francisco, CA 94080; Telephone: 415‐650‐6748; Fax: 888‐372‐8398; E‐mail: alex.dummett@kp.org

Content Gating

Gated (full article locked unless allowed per User)

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

Media Files

EMR‐Based Detection of Deterioration

Article Type

Changed

Mon, 01/30/2017 - 11:16

Display Headline

Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals

Author(s)

Benjamin J. Turk, BS, MAS

APPENDIX 1: EXPANDED DESCRIPTION OF VARIABLES INCLUDED IN THE PREDICTIVE MODEL APPENDIX 2: PREDICTOR VARIABLES – EXPANDED DESCRIPTION APPENDIX 3: INSTANTIATION APPROACHES WE CONSIDERED (1)

Patients who deteriorate in the hospital and are transferred to the intensive care unit (ICU) have higher mortality and greater morbidity than those directly admitted from the emergency department.[1, 2, 3] Rapid response teams (RRTs) were created to address this problem.[4, 5] Quantitative tools, such as the Modified Early Warning Score (MEWS),[6] have been used to support RRTs almost since their inception. Nonetheless, work on developing scores that can serve as triggers for RRT evaluation or intervention continues. The notion that comprehensive inpatient electronic medical records (EMRs) could support RRTs (both as a source of patient data and a platform for providing alerts) has intuitive appeal. Not surprisingly, in addition to newer versions of manual scores,[7] electronic scores are now entering clinical practice. These newer systems are being tested in research institutions,[8] hospitals with advanced capabilities,[9] and as part of proprietary systems.[10] Although a fair amount of statistical information (eg, area under the receiver operator characteristic curve of a given predictive model) on the performance of various trigger systems has been published, existing reports have not described details of how the electronic architecture is integrated with clinical practice.

Electronic alert systems generated from physiology‐based predictive models do not yet constitute mature technologies. No consensus or legal mandate regarding their role yet exists. Given this situation, studying different implementation approaches and their outcomes has value. It is instructive to consider how a given institutional solution addresses common contingenciesoperational constraints that are likely to be present, albeit in different forms, in most placesto help others understand the limitations and issues they may present. In this article we describe the structure of an EMR‐based early warning system in 2 pilot hospitals at Kaiser Permanente Northern California (KPNC). In this pilot, we embedded an updated version of a previously described early warning score[11] into the EMR. We will emphasize how its components address institutional, operational, and technological constraints. Finally, we will also describe unfinished businesschanges we would like to see in a future dissemination phase. Two important aspects of the pilot (development of a clinical response arm and addressing patient preferences with respect to supportive care) are being described elsewhere in this issue of the Journal of Hospital Medicine. Analyses of the actual impact on patient outcomes will be reported elsewhere; initial results appear favorable.[12]

INITIAL CONSTRAINTS

The ability to actually prevent inpatient deteriorations may be limited,[13] and doubts regarding the value of RRTs persist.[14, 15, 16] Consequently, work that led to the pilot occurred in stages. In the first stage (prior to 2010), our team presented data to internal audiences documenting the rates and outcomes of unplanned transfers from the ward to the ICU. Concurrently, our team developed a first generation risk adjustment methodology that was published in 2008.[17] We used this methodology to show that unplanned transfers did, in fact, have elevated mortality, and that this persisted after risk adjustment.[1, 2, 3] This phase of our work coincided with KPNC's deployment of the Epic inpatient EMR (www.epicsystems.com), known internally as KP HealthConnect [KPHC]), which was completed in 2010. Through both internal and external funding sources, we were able to create infrastructure to acquire clinical data, develop a prototype predictive model, and demonstrate superiority over manually assigned scores such as the MEWS.[11] Shortly thereafter, we developed a new risk adjustment capability.[18] This new capability includes a generic severity of illness score (Laboratory‐based Acute Physiology Score, version 2 [LAPS2]) and a longitudinal comorbidity score (Comorbidity Point Score, version 2 [COPS2]). Both of these scores have multiple uses (eg, for prediction of rehospitalization[19]) and are used for internal benchmarking at KPNC.

Once we demonstrated that we could, in fact, predict inpatient deteriorations, we still had to address medicallegal considerations, the need for a clinical response arm, and how to address patient preferences with respect to supportive or palliative care. To address these concerns and ensure that the implementation would be seamlessly integrated with routine clinical practice, our team worked for 1 year with hospitalists and other clinicians at the pilot sites prior to the go‐live date.

The primary concern from a medicallegal perspective is that once results from a predictive model (which could be an alert, severity score, comorbidity score, or other probability estimate) are displayed in the chart, relevant clinical information has been changed. Thus, failure to address such an EMR item could lead to malpractice risk for individuals and/or enterprise liability for an organization. After discussing this with senior leadership, they specified that it would be permissible to go forward so long as we could document that an educational intervention was in place to make sure that clinicians understood the system and that it was linked to specific protocols approved by hospitalists.

Current predictive models, including ours, generate a probability estimate. They do not necessarily identify the etiology of a problem or what solutions ought to be considered. Consequently, our senior leadership insisted that we be able to answer clinicians' basic question: What do we do when we get an alert? The article by Dummett et al.[20] in this issue of the Journal of Hospital Medicine describes how we addressed this constraint. Lastly, not all patients can be rescued. The article by Granich et al.[21] describes how we handled the need to respect patient choices.

PROCEDURAL COMPONENTS

The Gordon and Betty Moore Foundation, which funded the pilot, only had 1 restriction (inclusion of a hospital in the Sacramento, California area). The other site was selected based on 2 initial criteria: (1) the chosen site had to be 1 of the smaller KPNC hospitals, and (2) the chosen site had to be easily accessible for the lead author (G.J.E.). The KPNC South San Francisco hospital was selected as the alpha site and the KPNC Sacramento hospital as the beta site. One of the major drivers for these decisions was that both had robust palliative care services. The Sacramento hospital is a larger hospital with a more complex caseload.

Prior to the go‐live dates (November 19, 2013 for South San Francisco and April 16, 2014 for Sacramento), the executive committees at both hospitals reviewed preliminary data and the implementation plans for the early warning system. Following these reviews, the executive committees approved the deployment. Also during this phase, in consultation with our communications departments, we adopted the name Advance Alert Monitoring (AAM) as the outward facing name for the system. We also developed recommended scripts for clinical staff to employ when approaching a patient in whom an alert had been issued (this is because the alert is calibrated so as to predict increased risk of deterioration within the next 12 hours, which means that a patient might be surprised as to why clinicians were suddenly evaluating them). Facility approvals occurred approximately 1 month prior to the go‐live date at each hospital, permitting a shadowing phase. In this phase, selected physicians were provided with probability estimates and severity scores, but these were not displayed in the EMR front end. This shadowing phase permitted clinicians to finalize the response arms' protocols that are described in the articles by Dummett et al.[20] and Granich et al.[21] We obtained approval from the KPNC Institutional Review Board for the Protection of Human Subjects for the evaluation component that is described below.

EARLY DETECTION ALGORITHMS

The early detection algorithms we employed, which are being updated periodically, were based on our previously published work.[11, 18] Even though admitting diagnoses were found to be predictive in our original model, during actual development of the real‐time data extraction algorithms, we found that diagnoses could not be obtained reliably, so we made the decision to use a single predictive equation for all patients. The core components of the AAM score equation are the above‐mentioned LAPS2 and COPS2; these are combined with other data elements (Table 1). None of the scores are proprietary, and our equations could be replicated by any entity with a comprehensive inpatient EMR. Our early detection system is calibrated using outcomes that occurred 12 hours from when the alert is issued. For prediction, it uses data from the preceding 12 months for the COPS2 and the preceding 24 to 72 hours for physiologic data.

Table 1.Variables Employed in Predictive Equation
Category	Elements Included	Comment
Demographics	Age, sex
Patient location	Unit indicators (eg, 3 West); also known as bed history indicators	Only patients in general medicalsurgical ward, transitional care unit, and telemetry unit are eligible. Patients in the operating room, postanesthesia recovery room, labor and delivery service, and pediatrics are ineligible.
Health services	Admission venue	Emergency department admission or not.
Health services	Elapsed length of stay in hospital up to the point when data are scanned	Interhospital transport is common in our integrated delivery system; this data element requires linking both unit stays as well as stays involving different hospitals.
Status	Care directive orders	Patients with a comfort careonly order are not eligible; all other patients (full code, partial code, and do not resuscitate) are.
Status	Admission status	Inpatients and patients admitted for observation status are eligible.
Physiologic	Vital signs, laboratory tests, neurological status checks	See online Appendices and references [6] and [15] for details on how we extract, format, and transform these variables.
Composite indices	Generic severity of illness score	See text and description in reference [15] for details on the Laboratory‐based Acute Physiology score, version 2 and the Comorbidity Point Score, version 2.
Composite indices	Longitudinal comorbidity score

During the course of developing the real‐time extraction algorithms, we encountered a number of delays in real‐time data acquisition. These fall into 2 categories: charting delay and server delay. Charting delay is due to nonautomated charting of vital signs by nurses (eg, a nurse obtains vital signs on a patient, writes them down on paper, and then enters them later). In general, this delay was in the 15‐ to 30‐minute range, but occasionally was as high as 2 hours. Server delay, which was variable and ranged from a few minutes to (occasionally) 1 to 2 hours, is due to 2 factors. The first is that certain point of care tests were not always uploaded into the EMR immediately. This is because the testing units, which can display results to clinicians within minutes, must be physically connected to a computer for uploading results. The second is the processing time required for the system to cycle through hundreds of patient records in the context of a very large EMR system (the KPNC Epic build runs in 6 separate geographic instances, and our system runs in 2 of these). Figure 1 shows that each probability estimate thus has what we called an uncertainty period of 2 hours (the +2 hours addresses the fact that we needed to give clinicians a minimum time to respond to an alert). Given limited resources and the need to balance accuracy of the alerts, adequate lead time, the presence of an uncertainty period, and alert fatigue, we elected to issue alerts every 6 hours (with the exact timing based on facility preferences).

Time intervals involved in real‐time capture and reporting of data from an inpatient electronic medical record. T0 refers to the time when data extraction occurs and the system's Java application issues a probability estimate. The figure shows that, because of charting and server delays, data may be delayed up to 2 hours. Similarly, because ∼2 hours may be required to mount a coherent clinical response, a total time period of ∼4 hours (uncertainty window) exists for a given probability estimate.

A summary of the components of our equation is provided in the Supporting Information, Appendices, in the online version of this article. The statistical performance characteristics of our final equation, which are based on approximately 262 million individual data points from 650,684 hospitalizations in which patients experienced 20,471 deteriorations, is being reported elsewhere. Between November 19, 2013 and November 30, 2015 (the most recent data currently available to us for analysis), a total of 26,386 patients admitted to the ward or transitional care unit at the 2 pilot sites were scored by the AAM system, and these patients generated 3,881 alerts involving a total of 1,413 patients, which meant an average of 2 alerts per day at South San Francisco and 4 alerts per day in Sacramento. Resource limitations have precluded us from conducting formal surveys to assess clinician acceptance. However, repeated meetings with both hospitalists as well as RRT nurses indicated that favorable departmental consensus exists.

INSTANTIATION OF ALGORITHMS IN THE EMR

Given the complexity of the calculations involving many variables (Table 1), we elected to employ Web services to extract data for processing using a Java application outside the EMR, which then pushed results into the EMR front end (Figure 2). Additional details on this decision are provided in the Supporting Information, Appendices, in the online version of this article. Our team had to expend considerable resources and time to map all necessary data elements in the real time environment, whose identifying characteristics are not the same as those employed by the KPHC data warehouse. Considerable debugging was required during the first 7 months of the pilot. Troubleshooting for the application was often required on very short notice (eg, when the system unexpectedly stopped issuing alerts during a weekend, or when 1 class of patients suddenly stopped receiving scores). It is likely that future efforts to embed algorithms in EMRs will experience similar difficulties, and it is wise to budget so as maximize available analytic and application programmer resources.

Overall system architecture. Raw data are extracted directly from the inpatient electronic medical record (EMR) as well as other servers. In our case, the longitudinal comorbidity score is generated monthly outside the EMR by a department known as Decision Support (DS) which then stores the data in the Integrated Data Repository (IDR). Abbreviations: COPS2, Comorbidity Point Score, version 2; KPNC, Kaiser Permanente Northern California.

Figure 3 shows the final appearance of the graphical user interface at KPHC, which provides clinicians with 3 numbers: ADV ALERT SCORE (AAM score) is the probability of experiencing unplanned transfer within the next 12 hours, COPS is the COPS2, and LAPS is the LAPS2 assigned at the time a patient is placed in a hospital room. The current protocol in place is that the clinical response arm is triggered when the AAM score is 8.

Screen shot showing how early warning system outputs are displayed in clinicians' inpatient dashboard. ADV ALERT SCORE (AAM score) indicates the probability that a patient will require unplanned transfer to intensive care within the next 12 hours. COPS shows the Comorbidity Point Score, version 2 (see Escobar et al.[18] for details). LAPS shows the Laboratory‐based Acute Physiology Score, version 2 (see Escobar et al.[18] for details).

LIMITATIONS

One of the limitations of working with a commercial EMR in a large system, such as KPNC, is that of scalability. Understandably, the organization is reluctant to make changes in the EMR that will not ultimately be deployed across all hospitals in the system. Thus, any significant modification of the EMR or its associated workflows must, from the outset, be structured for subsequent spread to the remaining hospitals (19 in our case). Because we had not deployed a system like this before, we did not know what to expect and, had we known then what experience has taught us, our initial requests would have been different. Table 2 summarizes the major changes we would have made to our implementation strategy had we known then what we know now.

Table 2.Desirable Modifications to Early Warning System Based on Experience During the Pilot
Component	Status in Pilot Application	Desirable Changes
NOTE: Abbreviations: COPS2, Comorbidity Point Score, version 2; ICU, intensive care unit; KP, Kaiser Permanente; LAPS2, Laboratory‐based Acute Physiology score, version 2; TCU, transitional care unit.
Degree of disaster recovery support	System outages are handled on an ad hoc basis.	Same level of support as is seen in regular clinical systems (24/7 technical support).
Laboratory data feed	Web service.	It would be extremely valuable to have a definite answer about whether alternative data feeds would be faster and more reliable.
LAPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
LAPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
COPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
COPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
Alert response tracking	None is available.	Functionality that permits tracking what the status is of patients in whom an alert was issued (who responded, where it is charted, etc.)could be structured as a workbench report in KP HealthConnectvery important because of medical legal reasons.
Trending capability for scores	None is available.	Trending display available in same location where vital signs and laboratory test results are displayed.
Messaging capability	Not currently available.	Transmission of scores to rapid response team (or other designated first responder) via a smartphone, thus obviating the need for staff to check the inpatient dashboard manually every 6 hours.

EVALUATION STRATEGY

Due to institutional constraints, it is not possible for us to conduct a gold standard pilot using patient‐level randomization, as described by Kollef et al.[8] Consequently, in addition to using the pilot to surface specific implementation issues, we had to develop a parallel scoring system for capturing key data points (scores, outcomes) not just at the 2 pilot sites, but also at the remaining 19 KPNC hospitals. This required that we develop electronic tools that would permit us to capture these data elements continuously, both prospectively as well as retrospectively. Thus, to give an example, we developed a macro that we call LAPS2 any time that permits us to assign a retrospective severity score given any T₀. Our ultimate goal is to evaluate the system's deployment using a stepped wedge design[22] in which geographically contiguous clusters of 2 to 4 hospitals go live periodically. The silver standard (a cluster trial involving randomization at the individual hospital level[23]) is not feasible because KPNC hospitals span a very broad geographic area, and it is more resource intensive in a shorter time span. In this context, the most important output from a pilot such as this is to generate an estimate of likely impact; this estimate then becomes a critical component for power calculations for the stepped wedge.

Our ongoing evaluation has all the limitations inherent in the analysis of nonrandomized interventions. Because it only involves 2 hospitals, it is difficult to assess variation due to facility‐specific factors. Finally, because our priority was to avoid alert fatigue, the total number of patients who experience an alert is small, limiting available sample size. Given these constraints, we will employ a counterfactual method, multivariate matching,[24, 25, 26] so as to come as close as possible to simulating a randomized trial. To control for hospital‐specific factors, matching will be combined with difference‐in‐differences[27, 28] methodology. Our basic approach takes advantage of the fact that, although our alert system is currently running in 2 hospitals, it is possible for us to assign a retrospective alert to patients at all KPNC hospitals. Using multivariate matching techniques, we will then create a cohort in which each patient who received an alert is matched to 2 patients who are given a retrospective virtual alert during the same time period in control facilities. The pre‐ and postimplementation outcomes of pilot and matched controls are compared. The matching algorithms specify exact matches on membership status, whether or not the patient had been admitted to the ICU prior to the first alert, and whether or not the patient was full code at the time of an alert. Once potential matches are found using the above procedures, our algorithms seek the closest match for the following variables: age, alert probability, COPS2, and admission LAPS2. Membership status is important, because many individuals who are not covered by the Kaiser Foundation Health Plan, Inc., are hospitalized at KPNC hospitals. Because these nonmembers' postdischarge outcomes cannot be tracked, it is important to control for this variable in our analyses.

Our electronic evaluation strategy also can be used to quantify pilot effects on length of stay (total, after an alert, and ICU), rehospitalization, use of hospice, mortality, and cost. However, it is not adequate for the evaluation of whether or not patient preferences are respected. Consequently, we have also developed manual review instruments for structured electronic chart review (the coding form and manual are provided in the online Appendix of the article in this issue of Journal of Hospital Medicine by Granich et al.[21]). This review will focus on issues such as whether or not patients' surrogates were identified, whether goals of care were discussed, and so forth. In those cases where patients died in the hospital, we will also review whether death occurred after resuscitation, whether family members were present, and so forth.

As noted above and in Figure 1, charting delays can result in uncertainty periods. We have found that these delays can also result in discrepancies in which data extracted from the real time system do not match those extracted from the data warehouse. These discrepancies can complicate creation of analysis datasets, which in turn can lead to delays in completing analyses. Such delays can cause significant problems with stakeholders. In retrospect, we should have devoted more resources to ongoing electronic audits and to the development of algorithms that formally address charting delays.

LESSONS LEARNED AND THOUGHTS ON FUTURE DISSEMINATION

We believe that embedding predictive models in the EMR will become an essential component of clinical care. Despite resource limitations and having to work in a frontier area, we did 3 things well. We were able to embed a complex set of equations and display their outputs in a commercial EMR outside the research setting. In a setting where hospitalists could have requested discontinuation of the system, we achieved consensus that it should remain the standard of care. Lastly, as a result of this work, KPNC will be deploying this early warning system in all its hospitals, so our overall implementation and communication strategy has been sound.

Nonetheless, our road to implementation has been a bumpy one, and we have learned a number of valuable lessons that are being incorporated into our future work. They merit sharing with the broader medical community. Using the title of a song by Ricky SkaggsIf I Had It All Again to Dowe can summarize what we learned with 3 phrases: engage leadership early, provide simpler explanations, and embed the evaluation in the solution.

Although our research on risk adjustment and the epidemiology was known to many KPNC leaders and clinicians, our initial engagement focus was on connecting with hospital physicians and operational leaders who worked in quality improvement. In retrospect, the research team should have engaged with 2 different communities much soonerthe information technology community and that component of leadership that focused on the EMR and information technology issues. Although these 2 broad communities interact with operations all the time, they do not necessarily have regular contact with research developments that might affect both EMR as well as quality improvement operations simultaneously. Not seeking this early engagement probably slowed our work by 9 to 15 months, because of repeated delays resulting from our assumption that the information technology teams understood things that were clear to us but not to them. One major result of this at KPNC is that we now have a regular quarterly meeting between researchers and the EMR leadership. The goal of this regular meeting is to make sure that operational leaders and researchers contemplating projects with an informatics component communicate early, long before any consideration of implementation occurs.

Whereas the notion of providing early warning seems intuitive and simple, translating this into a set of equations is challenging. However, we have found that developing equations is much easier than developing communication strategies suitable for people who are not interested in statistics, a group that probably constitutes the majority of clinicians. One major result of this learning now guiding our work is that our team devotes more time to considering existing and possible workflows. This process includes spending more time engaging with clinicians around how they use information. We are also experimenting with different ways of illustrating statistical concepts (eg, probabilities, likelihood ratios).

As is discussed in the article by Dummett et al.,[20] 1 workflow component that remains unresolved is that of documentation. It is not clear what the documentation standard should be for a deterioration probability. Solving this particular conundrum is not something that can be done by electronic or statistical means. However, also with the benefit of hindsight, we now know that we should have put more energy into automated electronic tools that provide support for documentation after an alert. In addition to being requested by clinicians, having tools that automatically generate tracers as part of both the alerting and documentation process would also make evaluation easier. For example, it would permit a better delineation of the causal path between the intervention (providing a deterioration probability) and patient outcomes. In future projects, incorporation of such tools will get much more prominence.

Acknowledgements

The authors thank Dr. Michelle Caughey, Dr. Philip Madvig, Dr. Patricia Conolly, and Ms. Barbara Crawford for their administrative support, Dr. Tracy Lieu for reviewing the manuscript, and Ms. Rachel Lesser for formatting the manuscript.

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Foundation and its staff played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. None of the sponsors had any involvement in our decision to submit this manuscript or in the determination of its contents. None of the authors has any conflicts of interest to declare of relevance to this work

Files

References

Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2012;8(1):13–19.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency team. Anaesth Intensive Care. 1995;23(2):183–186.
Goldhill DR. The critically ill: following your MEWS. QJM. 2001;94(10):507–510.
National Health Service. National Early Warning Score (NEWS). Standardising the Assessment Of Acute‐Illness Severity in the NHS. Report of a Working Party. London, United Kingdom: Royal College of Physicians; 2012.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.
Bradley EH, Yakusheva O, Horwitz LI, Sipsma H, Fletcher J. Identifying patients at increased risk for unplanned readmission. Med Care. 2013;51(9):761–766.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar G, Liu V, Kim YS, et al. Early detection of impending deterioration outside the ICU: a difference‐in‐differences (DiD) study. Presented at: American Thoracic Society International Conference, San Francisco, California; May 13–18, 2016; A7614.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Escobar G, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and post‐discharge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191.
Meurer WJ, Lewis RJ. Cluster randomized trials: evaluating treatments applied to groups. JAMA. 2015;313(20):2068–2069.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study. Eli Lilly working paper available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference‐in‐differences approach. JAMA. 2014;312(22):2401–2402.
Ryan AM, Burgess JF, Dimick JB. Why we should not be indifferent to specification choices for difference‐in‐differences. Health Serv Res. 2015;50(4):1211–1235.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S18-S24

Read more about EMR‐Based Detection of Deterioration

Sections

Author(s)

Benjamin J. Turk, BS, MAS

Author(s)

Benjamin J. Turk, BS, MAS

APPENDIX 1: EXPANDED DESCRIPTION OF VARIABLES INCLUDED IN THE PREDICTIVE MODEL APPENDIX 2: PREDICTOR VARIABLES – EXPANDED DESCRIPTION APPENDIX 3: INSTANTIATION APPROACHES WE CONSIDERED (1)

Files

APPENDIX 1: EXPANDED DESCRIPTION OF VARIABLES INCLUDED IN THE PREDICTIVE MODEL APPENDIX 2: PREDICTOR VARIABLES – EXPANDED DESCRIPTION APPENDIX 3: INSTANTIATION APPROACHES WE CONSIDERED (1)

Files

Article PDF

Article PDF

INITIAL CONSTRAINTS

PROCEDURAL COMPONENTS

EARLY DETECTION ALGORITHMS

Table 1.Variables Employed in Predictive Equation
Category	Elements Included	Comment
Demographics	Age, sex
Patient location	Unit indicators (eg, 3 West); also known as bed history indicators	Only patients in general medicalsurgical ward, transitional care unit, and telemetry unit are eligible. Patients in the operating room, postanesthesia recovery room, labor and delivery service, and pediatrics are ineligible.
Health services	Admission venue	Emergency department admission or not.
Health services	Elapsed length of stay in hospital up to the point when data are scanned	Interhospital transport is common in our integrated delivery system; this data element requires linking both unit stays as well as stays involving different hospitals.
Status	Care directive orders	Patients with a comfort careonly order are not eligible; all other patients (full code, partial code, and do not resuscitate) are.
Status	Admission status	Inpatients and patients admitted for observation status are eligible.
Physiologic	Vital signs, laboratory tests, neurological status checks	See online Appendices and references [6] and [15] for details on how we extract, format, and transform these variables.
Composite indices	Generic severity of illness score	See text and description in reference [15] for details on the Laboratory‐based Acute Physiology score, version 2 and the Comorbidity Point Score, version 2.
Composite indices	Longitudinal comorbidity score

INSTANTIATION OF ALGORITHMS IN THE EMR

LIMITATIONS

Table 2.Desirable Modifications to Early Warning System Based on Experience During the Pilot
Component	Status in Pilot Application	Desirable Changes
NOTE: Abbreviations: COPS2, Comorbidity Point Score, version 2; ICU, intensive care unit; KP, Kaiser Permanente; LAPS2, Laboratory‐based Acute Physiology score, version 2; TCU, transitional care unit.
Degree of disaster recovery support	System outages are handled on an ad hoc basis.	Same level of support as is seen in regular clinical systems (24/7 technical support).
Laboratory data feed	Web service.	It would be extremely valuable to have a definite answer about whether alternative data feeds would be faster and more reliable.
LAPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
LAPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
COPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
COPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
Alert response tracking	None is available.	Functionality that permits tracking what the status is of patients in whom an alert was issued (who responded, where it is charted, etc.)could be structured as a workbench report in KP HealthConnectvery important because of medical legal reasons.
Trending capability for scores	None is available.	Trending display available in same location where vital signs and laboratory test results are displayed.
Messaging capability	Not currently available.	Transmission of scores to rapid response team (or other designated first responder) via a smartphone, thus obviating the need for staff to check the inpatient dashboard manually every 6 hours.

EVALUATION STRATEGY

LESSONS LEARNED AND THOUGHTS ON FUTURE DISSEMINATION

Acknowledgements

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Foundation and its staff played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. None of the sponsors had any involvement in our decision to submit this manuscript or in the determination of its contents. None of the authors has any conflicts of interest to declare of relevance to this work

INITIAL CONSTRAINTS

PROCEDURAL COMPONENTS

EARLY DETECTION ALGORITHMS

Table 1.Variables Employed in Predictive Equation
Category	Elements Included	Comment
Demographics	Age, sex
Patient location	Unit indicators (eg, 3 West); also known as bed history indicators	Only patients in general medicalsurgical ward, transitional care unit, and telemetry unit are eligible. Patients in the operating room, postanesthesia recovery room, labor and delivery service, and pediatrics are ineligible.
Health services	Admission venue	Emergency department admission or not.
Health services	Elapsed length of stay in hospital up to the point when data are scanned	Interhospital transport is common in our integrated delivery system; this data element requires linking both unit stays as well as stays involving different hospitals.
Status	Care directive orders	Patients with a comfort careonly order are not eligible; all other patients (full code, partial code, and do not resuscitate) are.
Status	Admission status	Inpatients and patients admitted for observation status are eligible.
Physiologic	Vital signs, laboratory tests, neurological status checks	See online Appendices and references [6] and [15] for details on how we extract, format, and transform these variables.
Composite indices	Generic severity of illness score	See text and description in reference [15] for details on the Laboratory‐based Acute Physiology score, version 2 and the Comorbidity Point Score, version 2.
Composite indices	Longitudinal comorbidity score

INSTANTIATION OF ALGORITHMS IN THE EMR

LIMITATIONS

Table 2.Desirable Modifications to Early Warning System Based on Experience During the Pilot
Component	Status in Pilot Application	Desirable Changes
NOTE: Abbreviations: COPS2, Comorbidity Point Score, version 2; ICU, intensive care unit; KP, Kaiser Permanente; LAPS2, Laboratory‐based Acute Physiology score, version 2; TCU, transitional care unit.
Degree of disaster recovery support	System outages are handled on an ad hoc basis.	Same level of support as is seen in regular clinical systems (24/7 technical support).
Laboratory data feed	Web service.	It would be extremely valuable to have a definite answer about whether alternative data feeds would be faster and more reliable.
LAPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
LAPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
COPS2 score	Score appears only on ward or TCU patients.	Display for all hospitalized adults (include anyone 18 years and include ICU patients).
COPS2 score	Score appears only on inpatient physician dashboard.	Display scores in multiple dashboards (eg, emergency department dashboard).
Alert response tracking	None is available.	Functionality that permits tracking what the status is of patients in whom an alert was issued (who responded, where it is charted, etc.)could be structured as a workbench report in KP HealthConnectvery important because of medical legal reasons.
Trending capability for scores	None is available.	Trending display available in same location where vital signs and laboratory test results are displayed.
Messaging capability	Not currently available.	Transmission of scores to rapid response team (or other designated first responder) via a smartphone, thus obviating the need for staff to check the inpatient dashboard manually every 6 hours.

EVALUATION STRATEGY

LESSONS LEARNED AND THOUGHTS ON FUTURE DISSEMINATION

Acknowledgements

Disclosures: This work was supported by a grant from the Gordon and Betty Moore Foundation (Early Detection, Prevention, and Mitigation of Impending Physiologic Deterioration in Hospitalized Patients Outside Intensive Care: Phase 3, pilot), The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Foundation and its staff played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. Dr. Liu was supported by the National Institute for General Medical Sciences award K23GM112018. None of the sponsors had any involvement in our decision to submit this manuscript or in the determination of its contents. None of the authors has any conflicts of interest to declare of relevance to this work

References

Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2012;8(1):13–19.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency team. Anaesth Intensive Care. 1995;23(2):183–186.
Goldhill DR. The critically ill: following your MEWS. QJM. 2001;94(10):507–510.
National Health Service. National Early Warning Score (NEWS). Standardising the Assessment Of Acute‐Illness Severity in the NHS. Report of a Working Party. London, United Kingdom: Royal College of Physicians; 2012.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.
Bradley EH, Yakusheva O, Horwitz LI, Sipsma H, Fletcher J. Identifying patients at increased risk for unplanned readmission. Med Care. 2013;51(9):761–766.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar G, Liu V, Kim YS, et al. Early detection of impending deterioration outside the ICU: a difference‐in‐differences (DiD) study. Presented at: American Thoracic Society International Conference, San Francisco, California; May 13–18, 2016; A7614.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Escobar G, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and post‐discharge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191.
Meurer WJ, Lewis RJ. Cluster randomized trials: evaluating treatments applied to groups. JAMA. 2015;313(20):2068–2069.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study. Eli Lilly working paper available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference‐in‐differences approach. JAMA. 2014;312(22):2401–2402.
Ryan AM, Burgess JF, Dimick JB. Why we should not be indifferent to specification choices for difference‐in‐differences. Health Serv Res. 2015;50(4):1211–1235.

References

Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
Delgado MK, Liu V, Pines JM, Kipnis P, Gardner MN, Escobar GJ. Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system. J Hosp Med. 2012;8(1):13–19.
Hournihan F, Bishop G., Hillman KM, Dauffurn K, Lee A. The medical emergency team: a new strategy to identify and intervene in high‐risk surgical patients. Clin Intensive Care. 1995;6:269–272.
Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency team. Anaesth Intensive Care. 1995;23(2):183–186.
Goldhill DR. The critically ill: following your MEWS. QJM. 2001;94(10):507–510.
National Health Service. National Early Warning Score (NEWS). Standardising the Assessment Of Acute‐Illness Severity in the NHS. Report of a Working Party. London, United Kingdom: Royal College of Physicians; 2012.
Kollef MH, Chen Y, Heard K, et al. A randomized trial of real‐time automated clinical deterioration alerts sent to a rapid response team. J Hosp Med. 2014;9(7):424–429.
Evans RS, Kuttler KG, Simpson KJ, et al. Automated detection of physiologic deterioration in hospitalized patients. J Am Med Inform Assoc. 2015;22(2):350–360.
Bradley EH, Yakusheva O, Horwitz LI, Sipsma H, Fletcher J. Identifying patients at increased risk for unplanned readmission. Med Care. 2013;51(9):761–766.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar G, Liu V, Kim YS, et al. Early detection of impending deterioration outside the ICU: a difference‐in‐differences (DiD) study. Presented at: American Thoracic Society International Conference, San Francisco, California; May 13–18, 2016; A7614.
Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2011;6(2):68–72.
Winters BD, Pham J, Pronovost PJ. Rapid response teams—walk, don't run. JAMA. 2006;296(13):1645–1647.
Winters BD, Pham JC, Hunt EA, Guallar E, Berenholtz S, Pronovost PJ. Rapid response systems: a systematic review. Crit Care Med. 2007;35(5):1238–1243.
Litvak E, Pronovost PJ. Rethinking rapid response teams. JAMA. 2010;304(12):1375–1376.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Escobar GJ, Gardner M, Greene JG, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Escobar G, Ragins A, Scheirer P, Liu V, Robles J, Kipnis P. Nonelective rehospitalizations and post‐discharge mortality: predictive models suitable for use in real time. Med Care. 2015;53(11):916–923.
Dummett et al. J Hosp Med. 2016;11:000–000.
Granich et al. J Hosp Med. 2016;11:000–000.
Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–191.
Meurer WJ, Lewis RJ. Cluster randomized trials: evaluating treatments applied to groups. JAMA. 2015;313(20):2068–2069.
Gu XS, Rosenbaum PR. Comparison of multivariate matching methods: structures, distances, and algorithms. J Comput Graph Stat. 1993;2(4):405–420.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study. Eli Lilly working paper available at: http://www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf.
Stuart EA. Matching methods for causal inference: a review and a look forward. Stat Sci. 2010;25(1):1–21.
Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference‐in‐differences approach. JAMA. 2014;312(22):2401–2402.
Ryan AM, Burgess JF, Dimick JB. Why we should not be indifferent to specification choices for difference‐in‐differences. Health Serv Res. 2015;50(4):1211–1235.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S18-S24

Page Number

S18-S24

Publications

Publications

Article Type

Display Headline

Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals

Display Headline

Piloting electronic medical record–based early detection of inpatient deterioration in community hospitals

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Gabriel J. Escobar, MD, Regional Director for Hospital Operations Research, Division of Research, Kaiser Permanente Northern California, 2000 Broadway Avenue, 032 R01, Oakland, CA 94612; Telephone: 510‐891‐3502; Fax: 510‐891‐3508; E‐mail: gabriel.escobar@kp.org

Content Gating

Gated (full article locked unless allowed per User)

Alternative CME

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

APPENDIX 1: EXPANDED DESCRIPTION OF VARIABLES INCLUDED IN THE PREDICTIVE MODEL APPENDIX 2: PREDICTOR VARIABLES – EXPANDED DESCRIPTION APPENDIX 3: INSTANTIATION APPROACHES WE CONSIDERED (1)

Media Files

The Learning Hospital System

Article Type

Changed

Mon, 01/02/2017 - 19:34

Display Headline

Data that drive: Closing the loop in the learning hospital system

Author(s)

John W. Morehouse, MD

Jennifer M. Baker, MPH

John D. Greene, MA

In the landmark Best Care at Lower Cost report, the Institute of Medicine presents a compelling vision of a US healthcare system where science, information technology, incentives, and care culture are brought together seamlessly to produce high‐quality healthcare.[1] At the center of this transformation is the learning healthcare system, a system characterized by its ability to leverage data arising from care provision to drive rapid improvements in care delivery.[2] When steeped within the right organizational milieu, these data help to close the virtuous cycle of continuous learning moving from science to evidence to care and back to new science. The anticipated end result is a healthcare system that can provide Americans with superior care at lower cost.

Hospital‐based practitioners will recognize the inpatient setting as an ideal demonstration opportunity for continuous learning. Hospital care is costly, accounting for more than 30% of all US healthcare costs[3]; intensive care alone accounts for a notable proportion of the US gross domestic product.[4] Inpatient care is associated with significant mortality and morbidity, and its use is often greatly increased in patients' last days.[5, 6] Fortunately, the inpatient setting also offers an ideal opportunity to leverage high‐quality data to help inform and improve care. The digitization of medicine means that far more data are now available through electronic health records, medical devices, and tests.[7] This is particularly true for inpatients, for whom a large volume of data are produced even over relatively short hospital stays.

Whereas the challenge to improve hospital care is daunting, there is an incredible opportunity to advance the quality of inpatient care through realizing the vision of the learning hospital system. In the sections that follow, we use an object lessonsepsis care within hospitals of the Kaiser Permanente Northern California (KPNC) integrated healthcare delivery systemto evaluate the challenges and insights gleaned from working toward building a learning hospital system. Then, we describe further steps that could enhance the use of inpatient data to drive improved care.

THE FRAMEWORK OF A LEARNING HEALTHCARE SYSTEM

Best Care at Lower Cost notes a fundamental paradox in US healthcare: although we have witnessed a dramatic expansion in biomedical knowledge, innovative therapies and surgical procedures, and clinical treatments to extend survival, US healthcare persistently falls short on the basic dimensions of quality, outcomes, cost, and equity.[1] The proposed path forward lies in building the learning healthcare system, a system characterized by continuous knowledge development, improvement, and application. Figure 1 shows the critical nodes in the framework for continuous learning, which include: (1) the development of new scientific knowledge (science), (2) the translation of science into clinical evidence of efficacy (evidence), and (3) the application of efficacious interventions through effective care delivery (care). In healthcare today, transitions between these nodes are rife with missed or wasted opportunities like delays in applying high‐quality evidence or poorly managed insights arising from scientific discovery. If such opportunities could be recovered, however, the quality of healthcare could be improved dramatically.[8]

Schematic of a continuously learning healthcare system, adapted from the Institute of Medicine's Best Care at Lower Cost report.

The pursuit of continuous learning is aided by rapid changes in the quality and quantity of biomedical data available over the past decade, especially through the use of electronic health records, novel biomolecular tools, and digital sensors.[2, 7, 9] The Internet has ushered in a new era of data connectivity, for example, allowing for highly engaged communication between patients and providers as well as collaboration between professional or citizen scientists on data of unprecedented scale.[10] New methodologic approaches, including data mining and machine learning, increasingly leverage commodity hardware to conduct previously computationally intractable analyses.[9] Moreover, the development of domain ontologies fosters the discovery of meaningful insights from data of heterogeneous types.[11]

Ultimately, however, improvements in data alone are inadequate to achieve continuous learning. As shown in Figure 1, whereas data form the channels that allow for transitions from science to evidence to care, novel insights need to be steeped within the right culture, motivated by the right incentives, and supported by the right leaders.[1, 12] Within the sustainable learning healthcare system, knowledge generation feeds practice change with the support and guidance of system leadership; improved practice, in turn, generates new knowledge and completes the virtuous cycle of learning.

THE PROMISE OF CONTINUOUS LEARNING IN HOSPITAL SETTINGS

The hospital is an ideal setting in which to foster continuous learning because advances in inpatient care have the potential to substantially improve healthcare quality and value.[8] Americans were hospitalized roughly 37 million times in 2012; in total, these episodes cost $378 billion.[3] Over 700,000 patients die in US hospitals annually, with reports showing that many patients utilize greatly increased inpatient and critical care services near the end of their lives in a manner that appears misaligned with their preferences.[11, 13] Hospital care is also highly variable in quality and cost; this heterogeneity is not closely associated with improved outcomes.[14, 15] Preventable harm and medical injury occur commonly in hospitals and are now recognized to be a leading cause of inpatient death.[16] Finally, emerging research illuminates the substantial toll that acute care has on patients and families resulting in new comorbidity, functional or neuropsychiatric impairment, rehospitalization, and financial burden that persist long after patients are discharged.[17]

Fortunately, inpatient care also exhibits several qualities that improve the likelihood that continuous learning can be achieved. Although it is clear that hospitalizations occur within the arc of a patient's larger health trajectory, these distinct episodes offer the potential to observe patient trajectories and treatments evolving within relatively compressed time intervals; over that same interval, a large volume of data are produced. Stored within comprehensive electronic health records, these granular data now allow inpatient episodes to be digitally recapitulated with high fidelity, bolstering their use in driving care improvements.[18]

AN OBJECT LESSON IN THE LEARNING FRAMEWORK: SEPSIS CARE

Translating Science to Evidence in Sepsis

Although sepsis has attracted great attention in modern hospital care, sepsis was described long ago by Hippocrates to describe the process by which wounds fester.[19] Recast after the confirmation of germ theory, sepsis came to be known primarily as the blood poisoning resulting from pathogenic organisms.[20] However, with the advent of antibiotics, numerous scientific studies now recognize that sepsis actually results from the dysregulated host immune response to systemic infection, which can also cause organ dysfunction.[21] Based on this knowledge, landmark translational and clinical studies in the 2000s provided strong evidence that early identification of sepsis patients and aggressive infection control and resuscitation were associated with improved mortality (Figure 2, step 1).[22]

Closing the continuous learning loop in sepsis care. Schematic representation of the continuous learning hospital system generating new discovery in sepsis care.

Translating Evidence to Care in Sepsis at KPNC

In 2007, the leadership of KPNC initiated a regional effort to improve the quality of care and reduce the variability in performance at its medical centers (Table 1).[23] Reviewing data from nearly 1000 inpatientsthe last 50 consecutive hospital deaths from each of 19 medical centersa mortality diagnostic based on Institute for Healthcare Improvement recommendations[24] revealed that sepsis had a major impact on hospital outcomes. For example, even though sepsis patients were still relatively under‐recognized at the time, accounting for fewer than 3% of hospitalizations, they contributed to one‐quarter of hospital deaths. In light of these compelling data, senior regional leadership identified reducing sepsis mortality as a key performance improvement goal (Figure 2, step 2).

Table 1.Timeline of Elements in the Kaiser Permanente Northern California Sepsis Performance Improvement Effort
Time Period	Event Summary
NOTE: Specific elements related to data infrastructure and analysis are indicated in italics. Abbreviations: ARISE, Australasian Resuscitation in Sepsis Evaluation; CMS SEP‐1, Centers for Medicare and Medicaide Services Severe Sepsis and Septic Shock: Management Bundle; ED, emergency department; EGDT, early goal‐directed therapy; KP, Kaiser Permanente; ProCESS, Protocolized Care for Early Septic Shock; ProMISe, Protocolized Care for Early Septic Shock.
2007	Variability in hospital standardized mortality ratio observed, indicating an opportunity to drive improved outcomes.
	Initiation of staggered implementation of unified electronic medical record across all KP sites (starting in 2006 and ending in 2009).
Spring 2008	Mortality diagnostic chart review completed identifying sepsis and infection‐related causes as key factors in hospital outcomes.
May 2008	Regional Mortality Summit held with a focus on patient safety and mortality reduction efforts through performance improvement. Executive regional and local leadership alignment to focus on sepsis performance improvement.
Summer 2008	Sepsis Steering Committee evaluates best available evidence, develops treatment algorithms, and plans for medical center pilots.
Fall 2008	Pilot intervention deployed at 2 medical centers.
November 2008	First Regional Sepsis Summit: development of sepsis performance improvement playbook, training materials, implementation plans, and measurement strategy.
November 2008	All medical centers begin to form multidisciplinary sepsis teams and performance improvement committees, obtain equipment and supplies including assembly of a sepsis cart. Multidisciplinary teams included ED physician champion, ED nurse champion, improvement advisor, hospitalists, intensivists, quality improvement personnel, nurse educators, and even resident physicians.
January 2009	Performance data collection begins on EGDT processes and outcomes. Initiation of 2 key elements to enhance screening for and detection of sepsis: (1) concomitant ordering of serum lactic acid along with blood cultures, and (2) definition of lactate >2.0 as a critical lab value.
	Use of manual chart review for case finding and central database entry because of ongoing implementation of electronic medical record and limited sepsis‐specific data infrastructure.
March 2009	Regional train the trainer sessions occur and local educational spread efforts begin including: collaborative calls, in‐person training events, and medical center site visits.
August 2009	Grant funding from the Gordon and Betty Moore Foundation begins with a planned 2‐year duration providing funding for improvement advisors with performance improvement expertise and data infrastructure development.
November 2009	Second Regional Sepsis Summit. Identification of intermediate lactate sepsis patients having significant mortality.
January 2010	Initiate measurement of performance for intermediate lactate sepsis patients with a focus on lactate clearance as an outcome measure of interest.
2010	Development of an intranet Web‐based data abstraction tool to identify cases and auto‐populate specific fields for review. Facilities were responsible for review of cases at the local level to foster rapid feedback cycles for local performance improvement. Standardized data query tools were deployed to foster local medical center engagement and system‐level evaluation.
	Accompanying development of a sepsis performance improvement scorecard allowing for comparison of longitudinal performance metrics across all facilities. Scorecard elements included: proportion of lactates drawn following ED blood culture, EGDT‐specific bundle elements (ie, number of EGDT cases, antibiotics within 1 hour, first central venous pressure within 2 hours of EGDT start, target mean arterial pressure achievement), repeat lactate elements, balancing measures for central line placement (ie, pneumothorax, central line infection), and overall sepsis statistics.
April 2011	Third Regional Sepsis Summit. Refinement of EGDT bundle and further development of intermediate lactate bundle approach, including piloting specific treatment bundles targeting this population. Collaborative performance improvement environment in which successful strategies at 1 site were rapidly disseminated to other sites including the Sepsis Alert and the Sepsis Clock.
May 2012	Research analysis of fluid volume and lactate clearance in intermediate lactate sepsis population begins.
February 2013	Fourth Regional Sepsis Summit. Regional spread of intermediate lactate bundle including the use of fluids, antibiotics, and repeat lactate measurements.
May 2013	Research analysis of the contribution of sepsis to hospital deaths (within KP and in national sample) as well as post‐sepsis resource utilization and mortality
March 2014	Publication of ProCESS randomized clinical trial, requiring systemic reevaluation of EGDT‐based sepsis strategy. Subsequent publications of ARISE and ProMISe trials confirming findings from ProCESS. Updated approach under consideration and informally disseminated to practitioners.
October 2014	Updated sepsis treatment guidelines and data capture strategy fully implemented moving away from a catheter‐based strategy for all EGDT‐eligible patients.
October 2015	Sixth Regional Sepsis Summit held to adjust sepsis treatment and data measurement strategy to align more closely with CMS SEP‐1 guidelines.

Based on the principles of performance improvement methodology, clinical and operational leaders established an environment with aligned culture, incentives, and leadership around sepsis care. The effort was launched in late 2008 at a Sepsis Summit, bringing together a multidisciplinary group of stakeholders (eg, hospitalist, emergency department, and intensive care chiefs of staff and nursing managers; medical center and nursing executive and operational leadership) and providing sepsis care pathways based on the best available evidence.[23] Regional investments in the digital infrastructure to support implementation resulted in the provision of granular data within monthly sepsis scorecards quantifying each medical center's performance and trends for a diverse set of sepsis bundle metrics.

The resulting changes in sepsis care were substantial. For example, improved early recognition of infected patients meeting the criteria for sepsis resulted in large changes in the standardized diagnostic criteria used to label patients (Figure 3A). Implementing screening strategies using serum lactate testing for any patient receiving blood cultures resulted in a roughly 10‐fold increase in the use of lactate testing in the emergency department (Figure 3B). Earlier recognition of sepsis also increased the number of patients receiving early antibiotics and receiving central venous catheters for quantitative resuscitation.[23]

(A) Changes in the diagnosis patterns among infected patients following program‐wide implementation of a sepsis performance improvement project at 21 hospitals in the Kaiser Permanente Northern California system. The lower portion of the figure displays the proportion of infection diagnosis codes among all hospital admissions, grouped into mutually exclusive and hierarchical categories (sorted in order as sepsis, respiratory, genitourinary, abdominal, other, skin/soft tissue, device‐related, vascular, or CNS). For example, patients with a diagnosis code of sepsis and respiratory infection would be coded exclusively within the sepsis category; the proportion of all infected patients coded as having sepsis increased rapidly after the sepsis program implementation. The upper portion of the figure shows the overall proportion of all hospital admissions that had at least 1 infection diagnosis codes (red solid line); the dotted grey line at 30% shows that, over the same period, there was only a modest increase in the proportion of all hospitalized patients with infection. (B) Changes in the number and mean values of serum lactate laboratory testing conducted in Kaiser Permanente Northern California emergency departments before and after sepsis performance improvement program implementation. Each point indicates a half‐year total. The blue line shows the total number of emergency department lactate values drawn following sepsis program implementation; there was a rapid rise in the number of lactate values checked, which plateaued after 2010. The black line shows the mean value of lactates and confidence limits (dotted lines show 95% confidence intervals) over time; the mean value of lactates decreased significantly as the volume of lactate testing increased. Abbreviations: CI, confidence interval; CNS, central nervous system; ED, emergency department.

CLOSING THE LOOP TOWARD CONTINUOUS LEARNING IN SEPSIS

Leveraging timely and actionable data steeped within an aligned organizational milieu resulted in large‐scale changes across a heterogeneous set of hospitals. However, to realize the true vision of a learning hospital system, a looming question remained: Could the data generated as the byproduct of routine care now be used to complete the virtuous cycle and drive new scientific discovery (Figure 2, step 3)?

Confirming Concordance in the Impact of Sepsis Nationally

The heightened identification of sepsis patients through program implementation revealed that the impact of sepsis on hospital mortality was greater than originally estimated; based on improved patient identification, sepsis now accounted for upward of 1 in 2 hospital deaths.[25] This sobering statistic confirmed that the investments in standardizing best sepsis care following the mortality diagnostic were critical. However, were similar estimates of sepsis‐attributable mortality consistent outside of the KPNC system? To study this, we examined US hospitalizations occurring across >1000 hospitals and involving >6 million hospital stays to estimate corresponding prevalence.[25] In this national sample, sepsis contributed to as many as half of hospital deaths in the United States in 2010, lending strong support to ongoing international and state‐based efforts to improve sepsis care. These studies also paved the way to use these data drawn from our large sepsis population to inform updated international consensus definitions for sepsis and septic shock.[26, 27, 28]

Identifying New Avenues for Reducing the Toll of Sepsis

A major concern of sepsis program leaders was the prior findings that sepsis hospitalizations among Medicare beneficiaries were associated with substantial new cognitive and functional disability.[29] This lingering toll of sepsis had been termed a hidden public health disaster.[30] To further understand the posthospital impact of sepsis and to begin investigating new avenues to reduce this impact, a cohort of patients was followed for 1 year following sepsis hospitalization.[31] Over that period, nearly half of sepsis survivors were rehospitalized. When compared with their presepsis levels of healthcare utilization, middle‐aged and elderly sepsis patients experienced a 3‐fold increase in their days using facility‐based care. Subsequent studies in other populations outside of KPNC have confirmed these concerning findings, resulting in new efforts to address postsepsis survivorship care.[32, 33]

Phenotyping New Targets for Standardized Sepsis Care

At its outset, the sepsis improvement program applied the best available evidence to treat patients with the most severe forms of sepsisseptic shock. However, once the initial implementation phase had succeeded, clinicians and operational leaders quickly realized from the emerging data that there was a far larger group of sepsis patients for whom treatment guidelines were poorly defined.[25, 34, 35] These were severe sepsis patients with so‐called intermediate lactate values between 2 mmol/L and 4 mmol/L; they comprised a substantial proportion of all sepsis patients dying in the hospital. Using data generated from the routine care of sepsis patients treated across 21 hospitals, the sepsis leadership group was able to rapidly assemble a cohort of intermediate lactate sepsis patients up to 20‐ to 100‐fold larger than that reported in prior studies and evaluate their outcomes.[34, 35]

The data used to evaluate these intermediate lactate sepsis patients now spurred a new implementation program in 2013 for a group of patients in whom there was essentially no existing evidence to guide care. Rapidly implemented within a mature sepsis performance improvement program, evaluations at the 6‐month and 1‐year intervals demonstrated significant decreases in mortality.[36] Importantly, to allay the justified concerns of clinicians, these evaluations also clearly showed no evidence of harm from more aggressive fluid resuscitation (eg, increased transfer to intensive care, increased rates of mechanical ventilation). Again, driven by clinician input, subgroup analyses further revealed that the implementation program was only associated with reduced mortality in patients who could be at risk for iatrogenic fluid overload (ie, those with a history of congestive heart failure or chronic kidney disease).[36] Spurred by these provocative findings, operational and clinical leaders are currently considering how to guide future care in these patients, especially with the emerging use of noninvasive methods to quantify patients' fluid responsiveness.

PRINCIPLES FOR LEVERAGING DATA IN THE LEARNING HOSPITAL SYSTEM

The object lesson of using data to drive improved sepsis care and further new scientific discovery offers some important insights for continuous learning.

Building a Digital Infrastructure for Utilizing Granular Hospital Data

As described above, current transitions between the nodes of the learning framework are rife with missed opportunities. Perhaps one of the most glaring is the inability to use highly granular data already collected within the electronic health record (eg, trajectories and trends across vital signs or laboratory results, large‐scale medication administration records to evaluate multidrug interactions). An essential starting point for continuous learning is investing in the digital infrastructure to improve the use of data beyond traditional claims (administrative dataadmission source codes, disposition codes, diagnoses, and procedures). As shown in Table 2, the first key step is incorporating laboratory data into the quality assessment/emmprovement process. In addition, using these data to automate severity of illness and risk adjustment metrics fosters use of similar comparison cohorts across time or disease types.[18, 37, 38, 39, 40]

Table 2.Data Types Necessary to Actualize the Learning Hospital System Graded by Contents and the Degree of Difficulty Necessary to Access and Analyze the Data
Data Type	Contents	Degree of Difficulty in Accessing	Degree of Difficulty in Analyzing
Administrative	Traditional claims data, diagnostic or procedural codes	Low	Low to moderate
Standard cohort profiling	Limited instances of vitals signs, laboratory, diagnostic testing, or treatment data	Low to moderate	Low to moderate
Metrics reporting for care improvement	Standard cohort identification, aggregated achievement of treatment targets, scorecard dissemination	Moderate	Moderate
Advanced cohort profiling	Time series of physiologic data, inpatient triage and treatment data within short temporal intervals	Moderate to high	High
Research‐grade discovery	Data with breadth (representative sample size) and depth (highly granular physiologic and treatment data)	High	Very high
Patient‐reported outcomes	Quality of life, functional and cognitive disability	Very high	High

Employing Novel Methods to Address the Limitations of Using Real‐World Data

The rapid digitization of medicine through the use of electronic medical records offers tremendous opportunities to facilitate continuous learning. However, these opportunities are accompanied by important limitations.[41] Data collected as a byproduct of real‐world care can be vulnerable to many forms of bias and confounding, potentially clouding the validity and robustness of corresponding analytic results. Fortunately, advanced methods including causal inference are now used routinely to address some limitations.[42] In the context of a learning healthcare system, other opportunities for improved study design including cluster randomized trials or stepped wedge implementation can also be employed to preserve the statistical rigor of subsequent analyses.[43] Finally, emerging methods employing randomization through the electronic medical record alongside adaptive trial design offer great potential to increase the efficiency of continuous learning.[44]

Evaluating the Hospital as a Single System

Advances in contemporary hospital care require seamless transitions of patient care, screening strategies, and therapeutic approaches across multiple hospital domains and with diverse providers; these interventions also need to happen rapidly. Many traditional approaches to inpatient care have taken a bottom‐up approach (eg, studying a specific disease within a specific hospital ward like the intensive care unit) that have proven useful but may limit generalizability when applied to a real‐world hospital operating with Pareto optimality (ie, the trade‐off scenario where new resource allocation to 1 area also requires resource withdrawal from another area). In certain cases, an empiric approach, without initial preference for any specific ward or disease, can aid decision making by hospital operational and clinical leaders by providing a global picture of impact and value.

Focusing on Early Detection in Hospital Settings as Secondary Prevention

Once patients have been admitted to the hospital, a race against the clock begins. Each additional hour of hospitalization increases the risks of iatrogenic injury or medical harm manifested by immobility, disorientation and delirium, nosocomial infections, or medication errors, among others. In this context, detection systems that use granular hospital data to focus on the earliest detection of risk can aid critical approaches to secondary prevention (Although the hospitalization for sepsis cannot be avoided, careful attention to mobility can limit the risk of developing delirium. In turn, preventing delirium can limit the risk of new functional disability).

Contextualizing Hospital Care Within a Longitudinal Trajectory

Although we described the benefit of hospital episodes having well‐demarcated beginning and ending points, it remains essential to recognize that the harms associated with hospitalization extend well beyond discharge. In this context, hospitalizations can serve as waypoints in patients' health trajectories as well as an opportunity to achieve patient‐centered care including discussing and aligning goals of care with actual care provision. Furthermore, although we have seen steady declines in hospital mortality over time, it is highly likely that we will reach a nadir in mortality where additional metrics of hospital outcomes will need to include postdischarge events like readmission, long‐term mortality, quality of life, and the prevention of disability or decline.

CONCLUSION

Hospitalizations in the United States are costly and associated with high mortality and morbidity; the toll of hospitalization also extends well beyond hospital discharge. The promise of the learning hospital system has marked improvements in the quality of hospital care, especially where healthcare systems can steep critical investments in data and digital infrastructure within the right culture, incentives, and leadership. Where continuous learning is achieved, data generated during routine care offer the potential to yield new scientific discovery and drive further improvements in hospital care.

Disclosures

As part of our agreement with the Gordon and Betty Moore Foundation, we made a commitment to disseminate our findings in articles such as this one. However, the Foundation and its staff played no role in how we actually structured our articles, nor did they review or preapprove any of the manuscripts submitted as part of the dissemination component. None of the authors has any conflicts of interest to declare of relevance to this work, which was funded by a combination of funding from the Gordon and Betty Moore Foundation, The Permanente Medical Group, Inc., and Kaiser Foundation Hospitals, Inc. VXL was supported by NIH K23GM112018.

Files

References

Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2012.
Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high‐functioning Learning Health System. J Am Med Inform Assoc. 2015;22(1):43–50.
National Center for Health Statistics. Health, United States, 2014: With Special Feature on Adults Aged 55–64. Hyattsville, MD; 2015.
Halpern NA, Pastores SM. Critical care medicine in the United States 2000‐2005: an analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(1):65–71.
Goodman DC., Esty AR, Fisher ES, Chang C‐H. Trends and variation in end‐of‐life care for medicare beneficiaries with severe chronic illness. A report of the Dartmouth Atlas Project. Lebanon, NH: The Dartmouth Institute for Health Policy and Clinical Practice; 2011.
Teno JM, Gozalo PL, Bynum JP, et al. Change in end‐of‐life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009. JAMA. 2013;309(5):470–477.
Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. JAMA. 2014;311(24):2479–2480.
Milstein A. Code red and blue—safely limiting health care's GDP footprint. N Engl J Med. 2013;368(1):1–3.
Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–1352.
Kullenberg C, Kasperowski D. What is citizen science?—a scientometric meta‐analysis. PLoS One. 2016;11(1):e0147152.
Rubin DL, Shah NH, Noy NF. Biomedical ontologies: a functional perspective. Brief Bioinform. 2008;9(1):75–90.
Etheredge LM. Rapid learning: a breakthrough agenda. Health Aff (Millwood). 2014;33(7):1155–1162.
Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end‐of‐life care intensity explained by patient preferences?: a study of the US Medicare population. Med Care. 2007;45(5):386–393.
Bai G, Anderson GF. Extreme markup: the fifty US hospitals with the highest charge‐to‐cost ratios. Health Aff (Millwood). 2015;34(6):922–928.
Cooper Z, Craig S, Gaynor M, Reenen J. The price ain't right? Hospital prices and health spending on the privately insured. Health Care Pricing Project website. Available at: http://www.healthcarepricingproject.org/sites/default/files/pricing_variation_manuscript_0.pdf. Accessed February 15, 2016
James JT. A new, evidence‐based estimate of patient harms associated with hospital care. J Patient Saf. 2013;9(3):122–128.
Covinsky KE, Pierluissi E, Johnston CB. Hospitalization‐associated disability: “she was probably able to ambulate, but I'm not sure”. JAMA. 2011;306(16):1782–1793.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Abraham E. New definitions for sepsis and septic shock: continuing evolution but with much still to be done. JAMA. 2016;315(8):757–759.
Angus DC, Poll T. Severe sepsis and septic shock. N Engl J Med. 2013;369(21):2063.
Hotchkiss RS, Monneret G, Payen D. Sepsis‐induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–874.
Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11):483–493.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90–92.
Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):762–774.
Shankar‐Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):775–787.
Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):801–810.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502–507.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Increased 1‐year healthcare use in survivors of severe sepsis. Am J Respir Crit Care Med. 2014;190(1):62–69.
Jones TK, Fuchs BD, Small DS, et al. Post‐acute care use and hospital readmission after sepsis. Ann Am Thorac Soc. 2015;12(6):904–913.
Liu V, Morehouse JW, Soule J, Whippy A, Escobar GJ. Fluid volume, lactate values, and mortality in sepsis patients with intermediate lactate values. Ann Am Thorac Soc. 2013;10(5):466–473.
Puskarich MA, Illich BM, Jones AE. Prognosis of emergency department patients with suspected infection and intermediate lactate levels: a systematic review. J Crit Care. 2014;29(3):334–339.
Liu VX, Morehouse JW, Marelich GP, et al. Multicenter implementation of a treatment bundle for sepsis patients with intermediate lactate values. Am J Respir Crit Care Med. 2016;193(11):1264–1270.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Schneeweiss S. Learning from big health care data. N Engl J Med. 2014;370(23):2161–2163.
Gabriel SE, Normand SL. Getting the methods right—the foundation of patient‐centered outcomes research. N Engl J Med. 2012;367(9):787–790.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391.
Angus DC. Fusing randomized trials with big data: the key to self‐learning health care systems? JAMA. 2015;314(8):767–768.

Article PDF

Issue

Journal of Hospital Medicine - 11(1)

Publications

Page Number

S11-S17

Read more about The Learning Hospital System

Sections

Author(s)

John W. Morehouse, MD

Jennifer M. Baker, MPH

John D. Greene, MA

Author(s)

John W. Morehouse, MD

Jennifer M. Baker, MPH

John D. Greene, MA

Files

Files

Article PDF

Article PDF

THE FRAMEWORK OF A LEARNING HEALTHCARE SYSTEM

THE PROMISE OF CONTINUOUS LEARNING IN HOSPITAL SETTINGS

AN OBJECT LESSON IN THE LEARNING FRAMEWORK: SEPSIS CARE

Translating Science to Evidence in Sepsis

Translating Evidence to Care in Sepsis at KPNC

Table 1.Timeline of Elements in the Kaiser Permanente Northern California Sepsis Performance Improvement Effort
Time Period	Event Summary
NOTE: Specific elements related to data infrastructure and analysis are indicated in italics. Abbreviations: ARISE, Australasian Resuscitation in Sepsis Evaluation; CMS SEP‐1, Centers for Medicare and Medicaide Services Severe Sepsis and Septic Shock: Management Bundle; ED, emergency department; EGDT, early goal‐directed therapy; KP, Kaiser Permanente; ProCESS, Protocolized Care for Early Septic Shock; ProMISe, Protocolized Care for Early Septic Shock.
2007	Variability in hospital standardized mortality ratio observed, indicating an opportunity to drive improved outcomes.
	Initiation of staggered implementation of unified electronic medical record across all KP sites (starting in 2006 and ending in 2009).
Spring 2008	Mortality diagnostic chart review completed identifying sepsis and infection‐related causes as key factors in hospital outcomes.
May 2008	Regional Mortality Summit held with a focus on patient safety and mortality reduction efforts through performance improvement. Executive regional and local leadership alignment to focus on sepsis performance improvement.
Summer 2008	Sepsis Steering Committee evaluates best available evidence, develops treatment algorithms, and plans for medical center pilots.
Fall 2008	Pilot intervention deployed at 2 medical centers.
November 2008	First Regional Sepsis Summit: development of sepsis performance improvement playbook, training materials, implementation plans, and measurement strategy.
November 2008	All medical centers begin to form multidisciplinary sepsis teams and performance improvement committees, obtain equipment and supplies including assembly of a sepsis cart. Multidisciplinary teams included ED physician champion, ED nurse champion, improvement advisor, hospitalists, intensivists, quality improvement personnel, nurse educators, and even resident physicians.
January 2009	Performance data collection begins on EGDT processes and outcomes. Initiation of 2 key elements to enhance screening for and detection of sepsis: (1) concomitant ordering of serum lactic acid along with blood cultures, and (2) definition of lactate >2.0 as a critical lab value.
	Use of manual chart review for case finding and central database entry because of ongoing implementation of electronic medical record and limited sepsis‐specific data infrastructure.
March 2009	Regional train the trainer sessions occur and local educational spread efforts begin including: collaborative calls, in‐person training events, and medical center site visits.
August 2009	Grant funding from the Gordon and Betty Moore Foundation begins with a planned 2‐year duration providing funding for improvement advisors with performance improvement expertise and data infrastructure development.
November 2009	Second Regional Sepsis Summit. Identification of intermediate lactate sepsis patients having significant mortality.
January 2010	Initiate measurement of performance for intermediate lactate sepsis patients with a focus on lactate clearance as an outcome measure of interest.
2010	Development of an intranet Web‐based data abstraction tool to identify cases and auto‐populate specific fields for review. Facilities were responsible for review of cases at the local level to foster rapid feedback cycles for local performance improvement. Standardized data query tools were deployed to foster local medical center engagement and system‐level evaluation.
	Accompanying development of a sepsis performance improvement scorecard allowing for comparison of longitudinal performance metrics across all facilities. Scorecard elements included: proportion of lactates drawn following ED blood culture, EGDT‐specific bundle elements (ie, number of EGDT cases, antibiotics within 1 hour, first central venous pressure within 2 hours of EGDT start, target mean arterial pressure achievement), repeat lactate elements, balancing measures for central line placement (ie, pneumothorax, central line infection), and overall sepsis statistics.
April 2011	Third Regional Sepsis Summit. Refinement of EGDT bundle and further development of intermediate lactate bundle approach, including piloting specific treatment bundles targeting this population. Collaborative performance improvement environment in which successful strategies at 1 site were rapidly disseminated to other sites including the Sepsis Alert and the Sepsis Clock.
May 2012	Research analysis of fluid volume and lactate clearance in intermediate lactate sepsis population begins.
February 2013	Fourth Regional Sepsis Summit. Regional spread of intermediate lactate bundle including the use of fluids, antibiotics, and repeat lactate measurements.
May 2013	Research analysis of the contribution of sepsis to hospital deaths (within KP and in national sample) as well as post‐sepsis resource utilization and mortality
March 2014	Publication of ProCESS randomized clinical trial, requiring systemic reevaluation of EGDT‐based sepsis strategy. Subsequent publications of ARISE and ProMISe trials confirming findings from ProCESS. Updated approach under consideration and informally disseminated to practitioners.
October 2014	Updated sepsis treatment guidelines and data capture strategy fully implemented moving away from a catheter‐based strategy for all EGDT‐eligible patients.
October 2015	Sixth Regional Sepsis Summit held to adjust sepsis treatment and data measurement strategy to align more closely with CMS SEP‐1 guidelines.

CLOSING THE LOOP TOWARD CONTINUOUS LEARNING IN SEPSIS

Confirming Concordance in the Impact of Sepsis Nationally

Identifying New Avenues for Reducing the Toll of Sepsis

Phenotyping New Targets for Standardized Sepsis Care

PRINCIPLES FOR LEVERAGING DATA IN THE LEARNING HOSPITAL SYSTEM

The object lesson of using data to drive improved sepsis care and further new scientific discovery offers some important insights for continuous learning.

Building a Digital Infrastructure for Utilizing Granular Hospital Data

Table 2.Data Types Necessary to Actualize the Learning Hospital System Graded by Contents and the Degree of Difficulty Necessary to Access and Analyze the Data
Data Type	Contents	Degree of Difficulty in Accessing	Degree of Difficulty in Analyzing
Administrative	Traditional claims data, diagnostic or procedural codes	Low	Low to moderate
Standard cohort profiling	Limited instances of vitals signs, laboratory, diagnostic testing, or treatment data	Low to moderate	Low to moderate
Metrics reporting for care improvement	Standard cohort identification, aggregated achievement of treatment targets, scorecard dissemination	Moderate	Moderate
Advanced cohort profiling	Time series of physiologic data, inpatient triage and treatment data within short temporal intervals	Moderate to high	High
Research‐grade discovery	Data with breadth (representative sample size) and depth (highly granular physiologic and treatment data)	High	Very high
Patient‐reported outcomes	Quality of life, functional and cognitive disability	Very high	High

Employing Novel Methods to Address the Limitations of Using Real‐World Data

Evaluating the Hospital as a Single System

Focusing on Early Detection in Hospital Settings as Secondary Prevention

Contextualizing Hospital Care Within a Longitudinal Trajectory

CONCLUSION

Disclosures

THE FRAMEWORK OF A LEARNING HEALTHCARE SYSTEM

THE PROMISE OF CONTINUOUS LEARNING IN HOSPITAL SETTINGS

AN OBJECT LESSON IN THE LEARNING FRAMEWORK: SEPSIS CARE

Translating Science to Evidence in Sepsis

Translating Evidence to Care in Sepsis at KPNC

Table 1.Timeline of Elements in the Kaiser Permanente Northern California Sepsis Performance Improvement Effort
Time Period	Event Summary
NOTE: Specific elements related to data infrastructure and analysis are indicated in italics. Abbreviations: ARISE, Australasian Resuscitation in Sepsis Evaluation; CMS SEP‐1, Centers for Medicare and Medicaide Services Severe Sepsis and Septic Shock: Management Bundle; ED, emergency department; EGDT, early goal‐directed therapy; KP, Kaiser Permanente; ProCESS, Protocolized Care for Early Septic Shock; ProMISe, Protocolized Care for Early Septic Shock.
2007	Variability in hospital standardized mortality ratio observed, indicating an opportunity to drive improved outcomes.
	Initiation of staggered implementation of unified electronic medical record across all KP sites (starting in 2006 and ending in 2009).
Spring 2008	Mortality diagnostic chart review completed identifying sepsis and infection‐related causes as key factors in hospital outcomes.
May 2008	Regional Mortality Summit held with a focus on patient safety and mortality reduction efforts through performance improvement. Executive regional and local leadership alignment to focus on sepsis performance improvement.
Summer 2008	Sepsis Steering Committee evaluates best available evidence, develops treatment algorithms, and plans for medical center pilots.
Fall 2008	Pilot intervention deployed at 2 medical centers.
November 2008	First Regional Sepsis Summit: development of sepsis performance improvement playbook, training materials, implementation plans, and measurement strategy.
November 2008	All medical centers begin to form multidisciplinary sepsis teams and performance improvement committees, obtain equipment and supplies including assembly of a sepsis cart. Multidisciplinary teams included ED physician champion, ED nurse champion, improvement advisor, hospitalists, intensivists, quality improvement personnel, nurse educators, and even resident physicians.
January 2009	Performance data collection begins on EGDT processes and outcomes. Initiation of 2 key elements to enhance screening for and detection of sepsis: (1) concomitant ordering of serum lactic acid along with blood cultures, and (2) definition of lactate >2.0 as a critical lab value.
	Use of manual chart review for case finding and central database entry because of ongoing implementation of electronic medical record and limited sepsis‐specific data infrastructure.
March 2009	Regional train the trainer sessions occur and local educational spread efforts begin including: collaborative calls, in‐person training events, and medical center site visits.
August 2009	Grant funding from the Gordon and Betty Moore Foundation begins with a planned 2‐year duration providing funding for improvement advisors with performance improvement expertise and data infrastructure development.
November 2009	Second Regional Sepsis Summit. Identification of intermediate lactate sepsis patients having significant mortality.
January 2010	Initiate measurement of performance for intermediate lactate sepsis patients with a focus on lactate clearance as an outcome measure of interest.
2010	Development of an intranet Web‐based data abstraction tool to identify cases and auto‐populate specific fields for review. Facilities were responsible for review of cases at the local level to foster rapid feedback cycles for local performance improvement. Standardized data query tools were deployed to foster local medical center engagement and system‐level evaluation.
	Accompanying development of a sepsis performance improvement scorecard allowing for comparison of longitudinal performance metrics across all facilities. Scorecard elements included: proportion of lactates drawn following ED blood culture, EGDT‐specific bundle elements (ie, number of EGDT cases, antibiotics within 1 hour, first central venous pressure within 2 hours of EGDT start, target mean arterial pressure achievement), repeat lactate elements, balancing measures for central line placement (ie, pneumothorax, central line infection), and overall sepsis statistics.
April 2011	Third Regional Sepsis Summit. Refinement of EGDT bundle and further development of intermediate lactate bundle approach, including piloting specific treatment bundles targeting this population. Collaborative performance improvement environment in which successful strategies at 1 site were rapidly disseminated to other sites including the Sepsis Alert and the Sepsis Clock.
May 2012	Research analysis of fluid volume and lactate clearance in intermediate lactate sepsis population begins.
February 2013	Fourth Regional Sepsis Summit. Regional spread of intermediate lactate bundle including the use of fluids, antibiotics, and repeat lactate measurements.
May 2013	Research analysis of the contribution of sepsis to hospital deaths (within KP and in national sample) as well as post‐sepsis resource utilization and mortality
March 2014	Publication of ProCESS randomized clinical trial, requiring systemic reevaluation of EGDT‐based sepsis strategy. Subsequent publications of ARISE and ProMISe trials confirming findings from ProCESS. Updated approach under consideration and informally disseminated to practitioners.
October 2014	Updated sepsis treatment guidelines and data capture strategy fully implemented moving away from a catheter‐based strategy for all EGDT‐eligible patients.
October 2015	Sixth Regional Sepsis Summit held to adjust sepsis treatment and data measurement strategy to align more closely with CMS SEP‐1 guidelines.

CLOSING THE LOOP TOWARD CONTINUOUS LEARNING IN SEPSIS

Confirming Concordance in the Impact of Sepsis Nationally

Identifying New Avenues for Reducing the Toll of Sepsis

Phenotyping New Targets for Standardized Sepsis Care

PRINCIPLES FOR LEVERAGING DATA IN THE LEARNING HOSPITAL SYSTEM

The object lesson of using data to drive improved sepsis care and further new scientific discovery offers some important insights for continuous learning.

Building a Digital Infrastructure for Utilizing Granular Hospital Data

Table 2.Data Types Necessary to Actualize the Learning Hospital System Graded by Contents and the Degree of Difficulty Necessary to Access and Analyze the Data
Data Type	Contents	Degree of Difficulty in Accessing	Degree of Difficulty in Analyzing
Administrative	Traditional claims data, diagnostic or procedural codes	Low	Low to moderate
Standard cohort profiling	Limited instances of vitals signs, laboratory, diagnostic testing, or treatment data	Low to moderate	Low to moderate
Metrics reporting for care improvement	Standard cohort identification, aggregated achievement of treatment targets, scorecard dissemination	Moderate	Moderate
Advanced cohort profiling	Time series of physiologic data, inpatient triage and treatment data within short temporal intervals	Moderate to high	High
Research‐grade discovery	Data with breadth (representative sample size) and depth (highly granular physiologic and treatment data)	High	Very high
Patient‐reported outcomes	Quality of life, functional and cognitive disability	Very high	High

Employing Novel Methods to Address the Limitations of Using Real‐World Data

Evaluating the Hospital as a Single System

Focusing on Early Detection in Hospital Settings as Secondary Prevention

Contextualizing Hospital Care Within a Longitudinal Trajectory

CONCLUSION

Disclosures

References

Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2012.
Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high‐functioning Learning Health System. J Am Med Inform Assoc. 2015;22(1):43–50.
National Center for Health Statistics. Health, United States, 2014: With Special Feature on Adults Aged 55–64. Hyattsville, MD; 2015.
Halpern NA, Pastores SM. Critical care medicine in the United States 2000‐2005: an analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(1):65–71.
Goodman DC., Esty AR, Fisher ES, Chang C‐H. Trends and variation in end‐of‐life care for medicare beneficiaries with severe chronic illness. A report of the Dartmouth Atlas Project. Lebanon, NH: The Dartmouth Institute for Health Policy and Clinical Practice; 2011.
Teno JM, Gozalo PL, Bynum JP, et al. Change in end‐of‐life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009. JAMA. 2013;309(5):470–477.
Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. JAMA. 2014;311(24):2479–2480.
Milstein A. Code red and blue—safely limiting health care's GDP footprint. N Engl J Med. 2013;368(1):1–3.
Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–1352.
Kullenberg C, Kasperowski D. What is citizen science?—a scientometric meta‐analysis. PLoS One. 2016;11(1):e0147152.
Rubin DL, Shah NH, Noy NF. Biomedical ontologies: a functional perspective. Brief Bioinform. 2008;9(1):75–90.
Etheredge LM. Rapid learning: a breakthrough agenda. Health Aff (Millwood). 2014;33(7):1155–1162.
Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end‐of‐life care intensity explained by patient preferences?: a study of the US Medicare population. Med Care. 2007;45(5):386–393.
Bai G, Anderson GF. Extreme markup: the fifty US hospitals with the highest charge‐to‐cost ratios. Health Aff (Millwood). 2015;34(6):922–928.
Cooper Z, Craig S, Gaynor M, Reenen J. The price ain't right? Hospital prices and health spending on the privately insured. Health Care Pricing Project website. Available at: http://www.healthcarepricingproject.org/sites/default/files/pricing_variation_manuscript_0.pdf. Accessed February 15, 2016
James JT. A new, evidence‐based estimate of patient harms associated with hospital care. J Patient Saf. 2013;9(3):122–128.
Covinsky KE, Pierluissi E, Johnston CB. Hospitalization‐associated disability: “she was probably able to ambulate, but I'm not sure”. JAMA. 2011;306(16):1782–1793.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Abraham E. New definitions for sepsis and septic shock: continuing evolution but with much still to be done. JAMA. 2016;315(8):757–759.
Angus DC, Poll T. Severe sepsis and septic shock. N Engl J Med. 2013;369(21):2063.
Hotchkiss RS, Monneret G, Payen D. Sepsis‐induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–874.
Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11):483–493.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90–92.
Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):762–774.
Shankar‐Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):775–787.
Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):801–810.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502–507.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Increased 1‐year healthcare use in survivors of severe sepsis. Am J Respir Crit Care Med. 2014;190(1):62–69.
Jones TK, Fuchs BD, Small DS, et al. Post‐acute care use and hospital readmission after sepsis. Ann Am Thorac Soc. 2015;12(6):904–913.
Liu V, Morehouse JW, Soule J, Whippy A, Escobar GJ. Fluid volume, lactate values, and mortality in sepsis patients with intermediate lactate values. Ann Am Thorac Soc. 2013;10(5):466–473.
Puskarich MA, Illich BM, Jones AE. Prognosis of emergency department patients with suspected infection and intermediate lactate levels: a systematic review. J Crit Care. 2014;29(3):334–339.
Liu VX, Morehouse JW, Marelich GP, et al. Multicenter implementation of a treatment bundle for sepsis patients with intermediate lactate values. Am J Respir Crit Care Med. 2016;193(11):1264–1270.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Schneeweiss S. Learning from big health care data. N Engl J Med. 2014;370(23):2161–2163.
Gabriel SE, Normand SL. Getting the methods right—the foundation of patient‐centered outcomes research. N Engl J Med. 2012;367(9):787–790.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391.
Angus DC. Fusing randomized trials with big data: the key to self‐learning health care systems? JAMA. 2015;314(8):767–768.

References

Institute of Medicine. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press; 2012.
Friedman C, Rubin J, Brown J, et al. Toward a science of learning systems: a research agenda for the high‐functioning Learning Health System. J Am Med Inform Assoc. 2015;22(1):43–50.
National Center for Health Statistics. Health, United States, 2014: With Special Feature on Adults Aged 55–64. Hyattsville, MD; 2015.
Halpern NA, Pastores SM. Critical care medicine in the United States 2000‐2005: an analysis of bed numbers, occupancy rates, payer mix, and costs. Crit Care Med. 2010;38(1):65–71.
Goodman DC., Esty AR, Fisher ES, Chang C‐H. Trends and variation in end‐of‐life care for medicare beneficiaries with severe chronic illness. A report of the Dartmouth Atlas Project. Lebanon, NH: The Dartmouth Institute for Health Policy and Clinical Practice; 2011.
Teno JM, Gozalo PL, Bynum JP, et al. Change in end‐of‐life care for Medicare beneficiaries: site of death, place of care, and health care transitions in 2000, 2005, and 2009. JAMA. 2013;309(5):470–477.
Weber GM, Mandl KD, Kohane IS. Finding the missing link for big biomedical data. JAMA. 2014;311(24):2479–2480.
Milstein A. Code red and blue—safely limiting health care's GDP footprint. N Engl J Med. 2013;368(1):1–3.
Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA. 2013;309(13):1351–1352.
Kullenberg C, Kasperowski D. What is citizen science?—a scientometric meta‐analysis. PLoS One. 2016;11(1):e0147152.
Rubin DL, Shah NH, Noy NF. Biomedical ontologies: a functional perspective. Brief Bioinform. 2008;9(1):75–90.
Etheredge LM. Rapid learning: a breakthrough agenda. Health Aff (Millwood). 2014;33(7):1155–1162.
Barnato AE, Herndon MB, Anthony DL, et al. Are regional variations in end‐of‐life care intensity explained by patient preferences?: a study of the US Medicare population. Med Care. 2007;45(5):386–393.
Bai G, Anderson GF. Extreme markup: the fifty US hospitals with the highest charge‐to‐cost ratios. Health Aff (Millwood). 2015;34(6):922–928.
Cooper Z, Craig S, Gaynor M, Reenen J. The price ain't right? Hospital prices and health spending on the privately insured. Health Care Pricing Project website. Available at: http://www.healthcarepricingproject.org/sites/default/files/pricing_variation_manuscript_0.pdf. Accessed February 15, 2016
James JT. A new, evidence‐based estimate of patient harms associated with hospital care. J Patient Saf. 2013;9(3):122–128.
Covinsky KE, Pierluissi E, Johnston CB. Hospitalization‐associated disability: “she was probably able to ambulate, but I'm not sure”. JAMA. 2011;306(16):1782–1793.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Abraham E. New definitions for sepsis and septic shock: continuing evolution but with much still to be done. JAMA. 2016;315(8):757–759.
Angus DC, Poll T. Severe sepsis and septic shock. N Engl J Med. 2013;369(21):2063.
Hotchkiss RS, Monneret G, Payen D. Sepsis‐induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–874.
Rivers E, Nguyen B, Havstad S, et al. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11):483–493.
Nadeem E, Olin SS, Hill LC, Hoagwood KE, Horwitz SM. Understanding the components of quality improvement collaboratives: a systematic literature review. Milbank Q. 2013;91(2):354–394.
Liu V, Escobar GJ, Greene JD, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA. 2014;312(1):90–92.
Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of clinical criteria for sepsis: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):762–774.
Shankar‐Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):775–787.
Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis‐3). JAMA. 2016;315(8):801–810.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Liu V, Lei X, Prescott HC, Kipnis P, Iwashyna TJ, Escobar GJ. Hospital readmission and healthcare utilization following sepsis in community settings. J Hosp Med. 2014;9(8):502–507.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Increased 1‐year healthcare use in survivors of severe sepsis. Am J Respir Crit Care Med. 2014;190(1):62–69.
Jones TK, Fuchs BD, Small DS, et al. Post‐acute care use and hospital readmission after sepsis. Ann Am Thorac Soc. 2015;12(6):904–913.
Liu V, Morehouse JW, Soule J, Whippy A, Escobar GJ. Fluid volume, lactate values, and mortality in sepsis patients with intermediate lactate values. Ann Am Thorac Soc. 2013;10(5):466–473.
Puskarich MA, Illich BM, Jones AE. Prognosis of emergency department patients with suspected infection and intermediate lactate levels: a systematic review. J Crit Care. 2014;29(3):334–339.
Liu VX, Morehouse JW, Marelich GP, et al. Multicenter implementation of a treatment bundle for sepsis patients with intermediate lactate values. Am J Respir Crit Care Med. 2016;193(11):1264–1270.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Schneeweiss S. Learning from big health care data. N Engl J Med. 2014;370(23):2161–2163.
Gabriel SE, Normand SL. Getting the methods right—the foundation of patient‐centered outcomes research. N Engl J Med. 2012;367(9):787–790.
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ. 2015;350:h391.
Angus DC. Fusing randomized trials with big data: the key to self‐learning health care systems? JAMA. 2015;314(8):767–768.

Issue

Journal of Hospital Medicine - 11(1)

Issue

Journal of Hospital Medicine - 11(1)

Page Number

S11-S17

Page Number

S11-S17

Publications

Publications

Article Type

Display Headline

Data that drive: Closing the loop in the learning hospital system

Display Headline

Data that drive: Closing the loop in the learning hospital system

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Vincent X. Liu, MD, 2000 Broadway, Oakland, CA 94612; Telephone: 510‐891‐5933; Fax: 510-627-2573; E‐mail: vincent.x.liu@kp.org

Content Gating

Gated (full article locked unless allowed per User)

Gating Strategy

First Peek Free

Teaser Media

Article PDF Media

Media Files

Healthcare Utilization after Sepsis

Article Type

Changed

Sun, 05/21/2017 - 14:04

Display Headline

Hospital readmission and healthcare utilization following sepsis in community settings

Author(s)

Xingye Lei, PhD, MA

Hallie C. Prescott, MD

Theodore J. Iwashyna, MD PhD

Sepsis, the systemic inflammatory response to infection, is a major public health concern.[1] Worldwide, sepsis affects millions of hospitalized patients each year.[2] In the United States, it is the single most expensive cause of hospitalization.[3, 4, 5, 6] Multiple studies suggest that sepsis hospitalizations are also increasing in frequency.[3, 6, 7, 8, 9, 10]

Improved sepsis care has dramatically reduced in‐hospital mortality.[11, 12, 13] However, the result is a growing number of sepsis survivors discharged with new disability.[1, 9, 14, 15, 16] Despite being a common cause of hospitalization, little is known about how to improve postsepsis care.[15, 17, 18, 19] This contrasts with other, often less common, hospital conditions for which many studies evaluating readmission and postdischarge care are available.[20, 21, 22, 23] Identifying the factors contributing to high utilization could lend critical insight to designing interventions that improve long‐term sepsis outcomes.[24]

We conducted a retrospective study of sepsis patients discharged in 2010 at Kaiser Permanente Northern California (KPNC) to describe their posthospital trajectories. In this diverse community‐hospitalbased population, we sought to identify the patient‐level factors that impact the posthospital healthcare utilization of sepsis survivors.

METHODS

This study was approved by the KPNC institutional review board.

Setting

We conducted a retrospective study of sepsis patients aged 18 years admitted to KPNC hospitals in 2010 whose hospitalizations included an overnight stay, began in a KPNC hospital, and was not for peripartum care. We identified sepsis based on International Classification of Disease, 9th Edition principal diagnosis codes used at KPNC, which capture a similar population to that from the Angus definition (see Supporting Appendix, Table 1, in the online version of this article).[7, 25, 26] We denoted each patient's first sepsis hospitalization as the index event.

Table 1.Baseline Patient and Hospital Characteristics of Patients With Sepsis Hospitalizations, Stratified by Predicted Hospital Mortality Quartiles
	Predicted Hospital Mortality Quartiles (n=1,586 for Each Group)
	Overall	1	2	3	4
NOTE: Data are presented as mean (standard deviation) or number (frequency). Abbreviations: COPS2: Comorbidity Point Score, version 2; ICU: intensive care unit; LAPS2: Laboratory Acute Physiology Score, version 2.
Baseline
Age, y, mean	71.915.7	62.317.8	71.214.2	75.612.7	78.612.2
By age category
<45 years	410 (6.5)	290 (18.3)	71 (4.5)	25 (1.6)	24 (1.5)
4564 years	1,425 (22.5)	539 (34.0)	407 (25.7)	292 (18.4)	187 (11.8)
6584 years	3,036 (47.9)	601 (37.9)	814 (51.3)	832 (52.5)	789 (49.8)
85 years	1,473 (23.2)	156 (9.8)	294 (18.5)	437 (27.6)	586 (37.0)
Male	2,973 (46.9)	686 (43.3)	792 (49.9)	750 (47.3)	745 (47.0)
Comorbidity
COPS2 score	5143	2627	5441	6445	6245
Charlson score	2.01.5	1.31.2	2.11.4	2.41.5	2.41.5
Hospitalization
LAPS2 severity score	10742	6621	9020	11423	15928
Admitted via emergency department	6,176 (97.4)	1,522 (96.0)	1,537 (96.9)	1,539 (97.0)	1,578 (99.5)
Direct ICU admission	1,730 (27.3)	169 (10.7)	309 (19.5)	482 (30.4)	770 (48.6)
ICU transfer, at any time	2,206 (34.8)	279 (17.6)	474 (29.9)	603 (38.0)	850 (53.6)
Hospital mortality
Predicted, %	10.513.8	1.00.1	3.40.1	8.32.3	29.415.8
Observed	865 (13.6)	26 (1.6)	86 (5.4)	197 (12.4)	556 (35.1)
Hospital length of stay, d	5.86.4	4.43.8	5.45.7	6.68.0	6.66.9

We linked hospital episodes with existing KPNC inpatient databases to describe patient characteristics.[27, 28, 29, 30] We categorized patients by age (45, 4564, 6584, and 85 years) and used Charlson comorbidity scores and Comorbidity Point Scores 2 (COPS2) to quantify comorbid illness burden.[28, 30, 31, 32] We quantified acute severity of illness using the Laboratory Acute Physiology Scores 2 (LAPS2), which incorporates 15 laboratory values, 5 vital signs, and mental status prior to hospital admission (including emergency department data).[30] Both the COPS2 and LAPS2 are independently associated with hospital mortality.[30, 31] We also generated a summary predicted risk of hospital mortality based on a validated risk model and stratified patients by quartiles.[30] We determined whether patients were admitted to the intensive care unit (ICU).[29]

Outcomes

We used patients' health insurance administrative data to quantify postsepsis utilization. Within the KPNC integrated healthcare delivery system, uniform information systems capture all healthcare utilization of insured members including services received at non‐KPNC facilities.[28, 30] We collected utilization data from the year preceding index hospitalization (presepsis) and for the year after discharge date or until death (postsepsis). We ascertained mortality after discharge from KPNC medical records as well as state and national death record files.

We grouped services into facility‐based or outpatient categories. Facility‐based services included inpatient admission, subacute nursing facility or long‐term acute care, and emergency department visits. We grouped outpatient services as hospice, home health, outpatient surgery, clinic, or other (eg, laboratory). We excluded patients whose utilization records were not available over the full presepsis interval. Among these 1211 patients (12.5% of total), the median length of records prior to index hospitalization was 67 days, with a mean value of 117 days.

Statistical Analysis

Our primary outcomes of interest were hospital readmission and utilization in the year after sepsis. We defined a hospital readmission as any inpatient stay after the index hospitalization grouped within 1‐, 3‐, 6‐, and 12‐month intervals. We designated those within 30 days as an early readmission. We grouped readmission principal diagnoses, where available, by the 17 Healthcare Cost and Utilization Project (HCUP) Clinical Classifications Software multilevel categories with sepsis in the infectious category.[33, 34] In secondary analysis, we also designated other infectious diagnoses not included in the standard HCUP infection category (eg, pneumonia, meningitis, cellulitis) as infection (see Supporting Appendix in the online version of this article).

We quantified outpatient utilization based on the number of episodes recorded. For facility‐based utilization, we calculated patient length of stay intervals. Because patients surviving their index hospitalization might not survive the entire year after discharge, we also calculated utilization adjusted for patients' living days by dividing the total facility length of stay by the number of living days after discharge.

Continuous data are represented as mean (standard deviation [SD]) and categorical data as number (%). We compared groups with analysis of variance or ² testing. We estimated survival with Kaplan‐Meier analysis (95% confidence interval) and compared groups with log‐rank testing. We compared pre‐ and postsepsis healthcare utilization with paired t tests.

To identify factors associated with early readmission after sepsis, we used a competing risks regression model.[35] The dependent variable was time to readmission and the competing hazard was death within 30 days without early readmission; patients without early readmission or death were censored at 30 days. The independent variables included age, gender, comorbid disease burden (COPS2), acute severity of illness (LAPS2), any use of intensive care, total index length of stay, and percentage of living days prior to sepsis hospitalization spent utilizing facility‐based care. We also used logistic regression to quantify the association between these variables and high postsepsis utilization; we defined high utilization as 15% of living days postsepsis spent in facility‐based care. For each model, we quantified the relative contribution of each predictor variable to model performance based on differences in log likelihoods.[35, 36] We conducted analyses using STATA/SE version 11.2 (StataCorp, College Station, TX) and considered a P value of <0.05 to be significant.

RESULTS

Cohort Characteristics

Our study cohort included 6344 patients with index sepsis hospitalizations in 2010 (Table 1). Mean age was 72 (SD 16) years including 1835 (28.9%) patients aged <65 years. During index hospitalizations, higher predicted mortality was associated with increased age, comorbid disease burden, and severity of illness (P<0.01 for each). ICU utilization increased across predicted mortality strata; for example, 10.7% of patients in the lowest quartile were admitted directly to the ICU compared with 48.6% in the highest quartile. In the highest quartile, observed mortality was 35.1%.

One‐Year Survival

A total of 5479 (86.4%) patients survived their index sepsis hospitalization. Overall survival after living discharge was 90.5% (range, 89.6%91.2%) at 30 days and 71.3% (range, 70.1%72.5%) at 1 year. However, postsepsis survival was strongly modified by age (Figure 1). For example, 1‐year survival was 94.1% (range, 91.2%96.0%) for <45 year olds and 54.4% (range, 51.5%57.2%) for 85 year olds (P<0.01). Survival was also modified by predicted mortality, however, not by ICU admission during index hospitalization (P=0.18) (see Supporting Appendix, Figure 1, in the online version of this article).

Kaplan‐Meier survival curves following living discharge after sepsis hospitalization, stratified by age categories.

Hospital Readmission

Overall, 978 (17.9%) patients had early readmission after index discharge (Table 2); nearly half were readmitted at least once in the year following discharge. Rehospitalization frequency was slightly lower when including patients with incomplete presepsis data (see Supporting Appendix, Table 2, in the online version of this article). The frequency of hospital readmission varied based on patient age and severity of illness. For example, 22.3% of patients in the highest predicted mortality quartile had early readmission compared with 11.6% in the lowest. The median time from discharge to early readmission was 11 days. Principal diagnoses were available for 78.6% of all readmissions (see Supporting Appendix, Table 3, in the online version of this article). Between 28.3% and 42.7% of those readmissions were for infectious diagnoses (including sepsis).

Table 2.Frequency of Readmissions After Surviving Index Sepsis Hospitalization, Stratified by Predicted Mortality Quartiles
		Predicted Mortality Quartile
Readmission	Overall	1	2	3	4
Within 30 days	978 (17.9)	158 (11.6)	242 (17.7)	274 (20.0)	304 (22.3)
Within 90 days	1,643 (30.1)	276 (20.2)	421 (30.8)	463 (33.9)	483 (35.4)
Within 180 days	2,061 (37.7)	368 (26.9)	540 (39.5)	584 (42.7)	569 (41.7)
Within 365 days	2,618 (47.9)	498 (36.4)	712 (52.1)	723 (52.9)	685 (50.2)

Table 3.Factors Associated With Early Readmission and High Postsepsis Facility‐Based Utilization
Variable	Hazard Ratio for Early Readmission		Odds Ratio for High Utilization
Variable	HR (95% CI)	Relative Contribution	OR (95% CI)	Relative Contribution
NOTE: High postsepsis utilization defined as 15% of living days spent in the hospital, subacute nursing facility, or long‐term acute care. Hazard ratios are based on competing risk regression, and odds ratios are based on logistic regression including all listed variables. Relative contribution to model performance was quantified by evaluating the differences in log likelihoods based on serial inclusion or exclusion of each variable. Abbreviations: CI, confidence interval; COPS2: Comorbidity Point Score, version 2; HR, hazard ratio; LAPS2: Laboratory Acute Physiology Score, version 2; OR, odds ratio. a P<0.01. b P<0.05.
Age category		1.2%		11.1%
<45 years	1.00 [reference]		1.00 [reference]
4564 years	0.86 (0.64‐1.16)		2.22 (1.30‐3.83)a
6584 years	0.92 (0.69‐1.21)		3.66 (2.17‐6.18)a
85 years	0.95 (0.70‐1.28)		4.98 (2.92‐8.50)a
Male	0.99 (0.88‐1.13)	0.0%	0.86 (0.74‐1.00)	0.1%
Severity of illness (LAPS2)	1.08 (1.04‐1.12)a	12.4%	1.22 (1.17‐1.27)a	11.3%
Comorbid illness (COPS2)	1.16 (1.12‐1.19)a	73.9%	1.13 (1.09‐1.17)a	5.9%
Intensive care	1.21 (1.05‐1.40)a	5.2%	1.02 (0.85‐1.21)	0.0%
Hospital length of stay, day	1.01 (1.001.02)b	6.6%	1.04 (1.03‐1.06)a	6.9%
Prior utilization, per 10%	0.98 (0.95‐1.02)	0.7%	1.74 (1.61‐1.88)a	64.2%

Healthcare Utilization

The unadjusted difference between pre‐ and postsepsis healthcare utilization among survivors was statistically significant for most categories but of modest clinical significance (see Supporting Appendix, Table 4, in the online version of this article). For example, the mean number of presepsis hospitalizations was 0.9 (1.4) compared to 1.0 (1.5) postsepsis (P<0.01). After adjusting for postsepsis living days, the difference in utilization was more pronounced (Figure 2). Overall, there was roughly a 3‐fold increase in the mean percentage of living days spent in facility‐based care between patients' pre‐ and postsepsis phases (5.3% vs 15.0%, P<0.01). Again, the difference was strongly modified by age. For patients aged <45 years, the difference was not statistically significant (2.4% vs 2.9%, P=0.32), whereas for those aged 65 years, it was highly significant (6.2% vs 18.5%, P<0.01).

Percentage of living days spent in facility‐based care, including inpatient hospitalization, subacute nursing facility, and long‐term acute care before and after index sepsis hospitalization.

Factors associated with early readmission included severity of illness, comorbid disease burden, index hospital length of stay, and intensive care (Table 3). However, the dominant factor explaining variation in the risk of early readmission was patients' prior comorbid disease burden (73.9%), followed by acute severity of illness (12.4%), total hospital length of stay (6.6%), and the need for intensive care (5.2%). Severity of illness and age were also significantly associated with higher odds of high postsepsis utilization; however, the dominant factor contributing to this risk was a history of high presepsis utilization (64.2%).

DISCUSSION

In this population‐based study in a community healthcare system, the impact of sepsis extended well beyond the initial hospitalization. One in 6 sepsis survivors was readmitted within 30 days, and roughly half were readmitted within 1 year. Fewer than half of rehospitalizations were for sepsis. Patients had a 3‐fold increase in the percentage of living days spent in hospitals or care facilities after sepsis hospitalization. Although age and acute severity of illness strongly modified healthcare utilization and mortality after sepsis, the dominant factors contributing to early readmission and high utilization ratescomorbid disease burden and presepsis healthcare utilizationwere present prior to hospitalization.

Sepsis is the single most expensive cause of US hospitalizations.[3, 4, 5] Despite its prevalence, there are little contemporary data identifying factors that impact healthcare utilization among sepsis survivors.[9, 16, 17, 19, 24, 36, 37] Recently, Prescott and others found that in Medicare beneficiaries, following severe sepsis, healthcare utilization was markedly increased.[17] More than one‐quarter of survivors were readmitted within 30 days, and 63.8% were readmitted within a year. Severe sepsis survivors also spent an average of 26% of their living days in a healthcare facility, a nearly 4‐fold increase compared to their presepsis phase. The current study included a population with a broader age and severity range; however, in a similar subgroup of patients, for those aged 65 years within the highest predicted mortality quartile, the frequency of readmission was similar. These findings are concordant with those from prior studies.[17, 19, 36, 37]

Among sepsis survivors, most readmissions were not for sepsis or infectious diagnoses, which is a novel finding with implications for designing approaches to reduce rehospitalization. The pattern in sepsis is similar to that seen in other common and costly hospital conditions.[17, 20, 23, 38, 39, 40] For example, between 18% and 25% of Medicare beneficiaries hospitalized for heart failure, acute myocardial infarction, or pneumonia were readmitted within 30 days; fewer than one‐third had the same diagnosis.[20] The timing of readmission in our sepsis cohort was also similar to that seen in other conditions.[20] For example, the median time of early readmission in this study was 11 days; it was between 10 and 12 days for patients with heart failure, pneumonia, and myocardial infarction.[20]

Krumholz and others suggest that the pattern of early rehospitalization after common acute conditions reflects a posthospital syndromean acquired, transient period of vulnerabilitythat could be the byproduct of common hospital factors.[20, 41] Such universal impairments might result from new physical and neurocognitive disability, nutritional deficiency, and sleep deprivation or delirium, among others.[41] If this construct were also true in sepsis, it could have important implications on the design of postsepsis care. However, prior studies suggest that sepsis patients may be particularly vulnerable to the sequelae of hospitalization.[2, 42, 43, 44, 45]

Among Medicare beneficiaries, Iwashyna and others reported that hospitalizations for severe sepsis resulted in significant increases in physical limitations and moderate to severe cognitive impairment.[1, 14, 46] Encephalopathy, sleep deprivation, and delirium are also frequently seen in sepsis patients.[47, 48] Furthermore, sepsis patients frequently need intensive care, which is also associated with increased patient disability and injury.[16, 46, 49, 50] We found that severity of illness and the need for intensive care were both predictive of the need for early readmission following sepsis. We also confirmed the results of prior studies suggesting that sepsis outcomes are strongly modified by age.[16, 19, 43, 51]

However, we found that the dominant factors contributing to patients' health trajectories were conditions present prior to admission. This finding is in accord with prior suggestions that acute severity of illness only partially predicts patients facing adverse posthospital sequelae.[23, 41, 52] Among sepsis patients, prior work demonstrates that inadequate consideration for presepsis level of function and utilization can result in an overestimation of the impact of sepsis on postdischarge health.[52, 53] Further, we found that the need for intensive care was not independently associated with an increased risk of high postsepsis utilization after adjusting for illness severity, a finding also seen in prior studies.[17, 23, 38, 51]

Taken together, our findings might suggest that an optimal approach to posthospital care in sepsis should focus on treatment approaches that address disease‐specific problems within the much larger context of common hospital risks. However, further study is necessary to clearly define the mechanisms by which age, severity of illness, and intensive care affect subsequent healthcare utilization. Furthermore, sepsis patients are a heterogeneous population in terms of severity of illness, site and pathogen of infection, and underlying comorbidity whose posthospital course remains incompletely characterized, limiting our ability to draw strong inferences.

These results should be interpreted in light of the study's limitations. First, our cohort included patients with healthcare insurance within a community‐based healthcare system. Care within the KPNC system, which bears similarities with accountable care organizations, is enhanced through service integration and a comprehensive health information system. Although prior studies suggest that these characteristics result in improved population‐based care, it is unclear whether there is a similar impact in hospital‐based conditions such as sepsis.[54, 55] Furthermore, care within an integrated system may impact posthospital utilization patterns and could limit generalizability. However, prior studies demonstrate the similarity of KPNC members to other patients in the same region in terms of age, socioeconomics, overall health behaviors, and racial/ethnic diversity.[56] Second, our study did not characterize organ dysfunction based on diagnosis coding, a common feature of sepsis studies that lack detailed physiologic severity data.[4, 5, 6, 8, 26] Instead, we focused on using granular laboratory and vital signs data to ensure accurate risk adjustment using a validated system developed in >400,000 hospitalizations.[30] Although this method may hamper comparisons with existing studies, traditional methods of grading severity by diagnosis codes can be vulnerable to biases resulting in wide variability.[10, 23, 26, 57, 58] Nonetheless, it is likely that characterizing preexisting and acute organ dysfunction will improve risk stratification in the heterogeneous sepsis population. Third, this study did not include data regarding patients' functional status, which has been shown to strongly predict patient outcomes following hospitalization. Fourth, this study did not address the cost of care following sepsis hospitalizations.[19, 59] Finally, our study excluded patients with incomplete utilization records, a choice designed to avoid the spurious inferences that can result from such comparisons.[53]

In summary, we found that sepsis exacted a considerable toll on patients in the hospital and in the year following discharge. Sepsis patients were frequently rehospitalized within a month of discharge, and on average had a 3‐fold increase in their subsequent time spent in healthcare facilities. Although age, severity of illness, and the need for ICU care impacted postsepsis utilization, the dominant contributing factorscomorbid disease burden or presepsis utilizationwere present prior to sepsis hospitalization. Early readmission patterns in sepsis appeared similar to those seen in other important hospital conditions, suggesting a role for shared posthospital, rather than just postsepsis, care approaches.

Disclosures

The funding for this study was provided by The Permanente Medical Group, Inc. and Kaiser Foundation Hospitals. The authors have no conflict of interests to disclose relevant to this article.

Files

References

Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Dellinger RP, Levy MM, Rhodes A, et al.; Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580–637.
Pfuntner A, Wier LM, Steiner C. Costs for hospital stays in the United States, 2010. HCUP statistical brief #16. January 2013. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb146.pdf. Accessed October 1, 2013.
Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546–1554.
Angus DC, Linde‐Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35(5):1244–1250.
Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. October 2011. Rockville, MD: Agency for Healthcare Research and Quality; 2011. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb122.pdf. Accessed October 1, 2013.
Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK. Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med. 2012;40(3):754–761.
Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population burden of long‐term survivorship after severe sepsis in older Americans. J Am Geriatr Soc. 2012;60(6):1070–1077.
Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41(5):1167–1174.
Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919–924.
Townsend SR, Schorr C, Levy MM, Dellinger RP. Reducing mortality in severe sepsis: the Surviving Sepsis Campaign. Clin Chest Med. 2008;29(4):721–733, x.
Rivers E, Nguyen B, Havstad S, et al.; Early Goal‐Directed Therapy Collaborative Group. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Winters BD, Eberlein M, Leung J, Needham DM, Pronovost PJ, Sevransky JE. Long‐term mortality and quality of life in sepsis: a systematic review. Crit Care Med. 2010;38(5):1276–1283.
Cuthbertson BH, Elders A, Hall S, et al.; the Scottish Critical Care Trials Group and the Scottish Intensive Care Society Audit Group. Mortality and quality of life in the five years after severe sepsis. Crit Care. 2013;17(2):R70.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Post‐Discharge Health Care Use Is Markedly Higher in Survivors of Severe Sepsis. Am J Respir Crit Care Med 2013;187:A1573.
Perl TM, Dvorak L, Hwang T, Wenzel RP. Long‐term survival and function after suspected gram‐negative sepsis. JAMA. 1995;274(4):338–345.
Weycker D, Akhras KS, Edelsberg J, Angus DC, Oster G. Long‐term mortality and medical care charges in patients with severe sepsis. Crit Care Med. 2003;31(9):2316–2323.
Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30‐day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA. 2013;309(4):355–363.
Gwadry‐Sridhar FH, Flintoft V, Lee DS, Lee H, Guyatt GH. A systematic review and meta‐analysis of studies comparing readmission rates and mortality rates in patients with heart failure. Arch Intern Med. 2004;164(21):2315–2320.
Gheorghiade M, Braunwald E. Hospitalizations for heart failure in the United States—a sign of hope. JAMA. 2011;306(15):1705–1706.
Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688–1698.
Iwashyna TJ, Odden AJ. Sepsis after Scotland: enough with the averages, show us the effect modifiers. Crit Care. 2013;17(3):148.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11): 483–493.
Iwashyna TJ, Odden A, Rohde J, et al. Identifying patients with severe sepsis using administrative claims: patient‐level validation of the Angus implementation of the International Consensus Conference Definition of Severe Sepsis [published online ahead of print September 18, 2012]. Med Care. doi: 10.1097/MLR.0b013e318268ac86. Epub ahead of print.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2009;63(7):798–803.
Cowen ME, Dusseau DJ, Toth BG, Guisinger C, Zodet MW, Shyr Y. Casemix adjustment of managed care claims data using the clinical classification for health policy research method. Med Care. 1998;36(7):1108–1113.
Agency for Healthcare Research and Quality Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD‐9‐CM Fact Sheet. Available at: http://www.hcup‐us.ahrq.gov/toolssoftware/ccs/ccsfactsheet.jsp. Accessed January 20, 2013.
Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1997;94(446):496–509.
Braun L, Riedel AA, Cooper LM. Severe sepsis in managed care: analysis of incidence, one‐year mortality, and associated costs of care. J Manag Care Pharm. 2004;10(6):521–530.
Lee H, Doig CJ, Ghali WA, Donaldson C, Johnson D, Manns B. Detailed cost analysis of care for survivors of severe sepsis. Crit Care Med. 2004;32(4):981–985.
Rico Crescencio JC, Leu M, Balaventakesh B, Loganathan R, et al. Readmissions among patients with severe sepsis/septic shock among inner‐city minority New Yorkers. Chest. 2012;142:286A.
Czaja AS, Zimmerman JJ, Nathens AB. Readmission and late mortality after pediatric severe sepsis. Pediatrics. 2009;123(3):849–857.
Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):1418–1428.
Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100–102.
Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–1655.
Martin GS, Mannino DM, Moss M. The effect of age on the development and outcome of adult sepsis. Crit Care Med. 2006;34(1):15–21.
Pinsky MR, Matuschak GM. Multiple systems organ failure: failure of host defense homeostasis. Crit Care Clin. 1989;5(2):199–220.
Remick DG. Pathophysiology of sepsis. Am J Pathol. 2007;170(5):1435–1444.
Angus DC, Carlet J. Surviving intensive care: a report from the 2002 Brussels Roundtable. Intensive Care Med. 2003;29(3):368–377.
Siami S, Annane D, Sharshar T. The encephalopathy in sepsis. Crit Care Clin. 2008;24(1):67–82, viii.
Gofton TE, Young GB. Sepsis‐associated encephalopathy. Nat Rev Neurol. 2012;8(10):557–566.
Needham DM, Davidson J, Cohen H, et al. Improving long‐term outcomes after discharge from intensive care unit: report from a stakeholders' conference. Crit Care Med. 2012;40(2):502–509.
Liu V, Turk BJ, Rizk NW, Kipnis P, Escobar GJ. The association between sepsis and potential medical injury among hospitalized patients. Chest. 2012;142(3):606–613.
Wunsch H, Guerra C, Barnato AE, Angus DC, Li G, Linde‐Zwirble WT. Three‐year outcomes for Medicare beneficiaries who survive intensive care. JAMA. 2010;303(9):849–856.
Clermont G, Angus DC, Linde‐Zwirble WT, Griffin MF, Fine MJ, Pinsky MR. Does acute organ dysfunction predict patient‐centered outcomes? Chest. 2002;121(6):1963–1971.
Iwashyna TJ, Netzer G, Langa KM, Cigolle C. Spurious inferences about long‐term outcomes: the case of severe sepsis and geriatric conditions. Am J Respir Crit Care Med. 2012;185(8):835–841.
Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Reed M, Huang J, Graetz I, et al., Outpatient electronic health records and the clinical care and outcomes of patients with diabetes mellitus. Ann Intern Med. 2012;157(7):482–489.
Gordon NP. Similarity of the adult Kaiser Permanente membership in Northern California to the insured and general population in Northern California: statistics from the 2009 California Health Interview Survey. Internal Division of Research Report. Oakland, CA: Kaiser Permanente Division of Research; January 24, 2012. Available at: http://www.dor.kaiser.org/external/chis_non_kp_2009. Accessed January 20, 2013.
Lindenauer PK, Lagu T, Shieh MS, Pekow PS, Rothberg MB. Association of diagnostic coding with trends in hospitalizations and mortality of patients with pneumonia, 2003–2009. JAMA. 2012;307(13):1405–1413.
Sarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012;307(13):1433–1435.
Kahn JM, Rubenfeld GD, Rohrbach J, Fuchs BD. Cost savings attributable to reductions in intensive care unit length of stay for mechanically ventilated patients. Med Care. 2008;46(12):1226–1233.

Article PDF

Issue

Journal of Hospital Medicine - 9(8)

Publications

Page Number

502-507

Read more about Healthcare Utilization after Sepsis

Sections

Author(s)

Xingye Lei, PhD, MA

Hallie C. Prescott, MD

Theodore J. Iwashyna, MD PhD

Author(s)

Xingye Lei, PhD, MA

Hallie C. Prescott, MD

Theodore J. Iwashyna, MD PhD

Files

Files

Article PDF

Article PDF

METHODS

This study was approved by the KPNC institutional review board.

Setting

Table 1.Baseline Patient and Hospital Characteristics of Patients With Sepsis Hospitalizations, Stratified by Predicted Hospital Mortality Quartiles
	Predicted Hospital Mortality Quartiles (n=1,586 for Each Group)
	Overall	1	2	3	4
NOTE: Data are presented as mean (standard deviation) or number (frequency). Abbreviations: COPS2: Comorbidity Point Score, version 2; ICU: intensive care unit; LAPS2: Laboratory Acute Physiology Score, version 2.
Baseline
Age, y, mean	71.915.7	62.317.8	71.214.2	75.612.7	78.612.2
By age category
<45 years	410 (6.5)	290 (18.3)	71 (4.5)	25 (1.6)	24 (1.5)
4564 years	1,425 (22.5)	539 (34.0)	407 (25.7)	292 (18.4)	187 (11.8)
6584 years	3,036 (47.9)	601 (37.9)	814 (51.3)	832 (52.5)	789 (49.8)
85 years	1,473 (23.2)	156 (9.8)	294 (18.5)	437 (27.6)	586 (37.0)
Male	2,973 (46.9)	686 (43.3)	792 (49.9)	750 (47.3)	745 (47.0)
Comorbidity
COPS2 score	5143	2627	5441	6445	6245
Charlson score	2.01.5	1.31.2	2.11.4	2.41.5	2.41.5
Hospitalization
LAPS2 severity score	10742	6621	9020	11423	15928
Admitted via emergency department	6,176 (97.4)	1,522 (96.0)	1,537 (96.9)	1,539 (97.0)	1,578 (99.5)
Direct ICU admission	1,730 (27.3)	169 (10.7)	309 (19.5)	482 (30.4)	770 (48.6)
ICU transfer, at any time	2,206 (34.8)	279 (17.6)	474 (29.9)	603 (38.0)	850 (53.6)
Hospital mortality
Predicted, %	10.513.8	1.00.1	3.40.1	8.32.3	29.415.8
Observed	865 (13.6)	26 (1.6)	86 (5.4)	197 (12.4)	556 (35.1)
Hospital length of stay, d	5.86.4	4.43.8	5.45.7	6.68.0	6.66.9

Outcomes

Statistical Analysis

RESULTS

Cohort Characteristics

One‐Year Survival

Hospital Readmission

Table 2.Frequency of Readmissions After Surviving Index Sepsis Hospitalization, Stratified by Predicted Mortality Quartiles
		Predicted Mortality Quartile
Readmission	Overall	1	2	3	4
Within 30 days	978 (17.9)	158 (11.6)	242 (17.7)	274 (20.0)	304 (22.3)
Within 90 days	1,643 (30.1)	276 (20.2)	421 (30.8)	463 (33.9)	483 (35.4)
Within 180 days	2,061 (37.7)	368 (26.9)	540 (39.5)	584 (42.7)	569 (41.7)
Within 365 days	2,618 (47.9)	498 (36.4)	712 (52.1)	723 (52.9)	685 (50.2)

Table 3.Factors Associated With Early Readmission and High Postsepsis Facility‐Based Utilization
Variable	Hazard Ratio for Early Readmission		Odds Ratio for High Utilization
Variable	HR (95% CI)	Relative Contribution	OR (95% CI)	Relative Contribution
NOTE: High postsepsis utilization defined as 15% of living days spent in the hospital, subacute nursing facility, or long‐term acute care. Hazard ratios are based on competing risk regression, and odds ratios are based on logistic regression including all listed variables. Relative contribution to model performance was quantified by evaluating the differences in log likelihoods based on serial inclusion or exclusion of each variable. Abbreviations: CI, confidence interval; COPS2: Comorbidity Point Score, version 2; HR, hazard ratio; LAPS2: Laboratory Acute Physiology Score, version 2; OR, odds ratio. a P<0.01. b P<0.05.
Age category		1.2%		11.1%
<45 years	1.00 [reference]		1.00 [reference]
4564 years	0.86 (0.64‐1.16)		2.22 (1.30‐3.83)a
6584 years	0.92 (0.69‐1.21)		3.66 (2.17‐6.18)a
85 years	0.95 (0.70‐1.28)		4.98 (2.92‐8.50)a
Male	0.99 (0.88‐1.13)	0.0%	0.86 (0.74‐1.00)	0.1%
Severity of illness (LAPS2)	1.08 (1.04‐1.12)a	12.4%	1.22 (1.17‐1.27)a	11.3%
Comorbid illness (COPS2)	1.16 (1.12‐1.19)a	73.9%	1.13 (1.09‐1.17)a	5.9%
Intensive care	1.21 (1.05‐1.40)a	5.2%	1.02 (0.85‐1.21)	0.0%
Hospital length of stay, day	1.01 (1.001.02)b	6.6%	1.04 (1.03‐1.06)a	6.9%
Prior utilization, per 10%	0.98 (0.95‐1.02)	0.7%	1.74 (1.61‐1.88)a	64.2%

Healthcare Utilization

DISCUSSION

Disclosures

The funding for this study was provided by The Permanente Medical Group, Inc. and Kaiser Foundation Hospitals. The authors have no conflict of interests to disclose relevant to this article.

METHODS

This study was approved by the KPNC institutional review board.

Setting

Table 1.Baseline Patient and Hospital Characteristics of Patients With Sepsis Hospitalizations, Stratified by Predicted Hospital Mortality Quartiles
	Predicted Hospital Mortality Quartiles (n=1,586 for Each Group)
	Overall	1	2	3	4
NOTE: Data are presented as mean (standard deviation) or number (frequency). Abbreviations: COPS2: Comorbidity Point Score, version 2; ICU: intensive care unit; LAPS2: Laboratory Acute Physiology Score, version 2.
Baseline
Age, y, mean	71.915.7	62.317.8	71.214.2	75.612.7	78.612.2
By age category
<45 years	410 (6.5)	290 (18.3)	71 (4.5)	25 (1.6)	24 (1.5)
4564 years	1,425 (22.5)	539 (34.0)	407 (25.7)	292 (18.4)	187 (11.8)
6584 years	3,036 (47.9)	601 (37.9)	814 (51.3)	832 (52.5)	789 (49.8)
85 years	1,473 (23.2)	156 (9.8)	294 (18.5)	437 (27.6)	586 (37.0)
Male	2,973 (46.9)	686 (43.3)	792 (49.9)	750 (47.3)	745 (47.0)
Comorbidity
COPS2 score	5143	2627	5441	6445	6245
Charlson score	2.01.5	1.31.2	2.11.4	2.41.5	2.41.5
Hospitalization
LAPS2 severity score	10742	6621	9020	11423	15928
Admitted via emergency department	6,176 (97.4)	1,522 (96.0)	1,537 (96.9)	1,539 (97.0)	1,578 (99.5)
Direct ICU admission	1,730 (27.3)	169 (10.7)	309 (19.5)	482 (30.4)	770 (48.6)
ICU transfer, at any time	2,206 (34.8)	279 (17.6)	474 (29.9)	603 (38.0)	850 (53.6)
Hospital mortality
Predicted, %	10.513.8	1.00.1	3.40.1	8.32.3	29.415.8
Observed	865 (13.6)	26 (1.6)	86 (5.4)	197 (12.4)	556 (35.1)
Hospital length of stay, d	5.86.4	4.43.8	5.45.7	6.68.0	6.66.9

Outcomes

Statistical Analysis

RESULTS

Cohort Characteristics

One‐Year Survival

Hospital Readmission

Table 2.Frequency of Readmissions After Surviving Index Sepsis Hospitalization, Stratified by Predicted Mortality Quartiles
		Predicted Mortality Quartile
Readmission	Overall	1	2	3	4
Within 30 days	978 (17.9)	158 (11.6)	242 (17.7)	274 (20.0)	304 (22.3)
Within 90 days	1,643 (30.1)	276 (20.2)	421 (30.8)	463 (33.9)	483 (35.4)
Within 180 days	2,061 (37.7)	368 (26.9)	540 (39.5)	584 (42.7)	569 (41.7)
Within 365 days	2,618 (47.9)	498 (36.4)	712 (52.1)	723 (52.9)	685 (50.2)

Table 3.Factors Associated With Early Readmission and High Postsepsis Facility‐Based Utilization
Variable	Hazard Ratio for Early Readmission		Odds Ratio for High Utilization
Variable	HR (95% CI)	Relative Contribution	OR (95% CI)	Relative Contribution
NOTE: High postsepsis utilization defined as 15% of living days spent in the hospital, subacute nursing facility, or long‐term acute care. Hazard ratios are based on competing risk regression, and odds ratios are based on logistic regression including all listed variables. Relative contribution to model performance was quantified by evaluating the differences in log likelihoods based on serial inclusion or exclusion of each variable. Abbreviations: CI, confidence interval; COPS2: Comorbidity Point Score, version 2; HR, hazard ratio; LAPS2: Laboratory Acute Physiology Score, version 2; OR, odds ratio. a P<0.01. b P<0.05.
Age category		1.2%		11.1%
<45 years	1.00 [reference]		1.00 [reference]
4564 years	0.86 (0.64‐1.16)		2.22 (1.30‐3.83)a
6584 years	0.92 (0.69‐1.21)		3.66 (2.17‐6.18)a
85 years	0.95 (0.70‐1.28)		4.98 (2.92‐8.50)a
Male	0.99 (0.88‐1.13)	0.0%	0.86 (0.74‐1.00)	0.1%
Severity of illness (LAPS2)	1.08 (1.04‐1.12)a	12.4%	1.22 (1.17‐1.27)a	11.3%
Comorbid illness (COPS2)	1.16 (1.12‐1.19)a	73.9%	1.13 (1.09‐1.17)a	5.9%
Intensive care	1.21 (1.05‐1.40)a	5.2%	1.02 (0.85‐1.21)	0.0%
Hospital length of stay, day	1.01 (1.001.02)b	6.6%	1.04 (1.03‐1.06)a	6.9%
Prior utilization, per 10%	0.98 (0.95‐1.02)	0.7%	1.74 (1.61‐1.88)a	64.2%

Healthcare Utilization

DISCUSSION

Disclosures

The funding for this study was provided by The Permanente Medical Group, Inc. and Kaiser Foundation Hospitals. The authors have no conflict of interests to disclose relevant to this article.

References

Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Dellinger RP, Levy MM, Rhodes A, et al.; Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580–637.
Pfuntner A, Wier LM, Steiner C. Costs for hospital stays in the United States, 2010. HCUP statistical brief #16. January 2013. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb146.pdf. Accessed October 1, 2013.
Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546–1554.
Angus DC, Linde‐Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35(5):1244–1250.
Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. October 2011. Rockville, MD: Agency for Healthcare Research and Quality; 2011. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb122.pdf. Accessed October 1, 2013.
Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK. Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med. 2012;40(3):754–761.
Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population burden of long‐term survivorship after severe sepsis in older Americans. J Am Geriatr Soc. 2012;60(6):1070–1077.
Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41(5):1167–1174.
Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919–924.
Townsend SR, Schorr C, Levy MM, Dellinger RP. Reducing mortality in severe sepsis: the Surviving Sepsis Campaign. Clin Chest Med. 2008;29(4):721–733, x.
Rivers E, Nguyen B, Havstad S, et al.; Early Goal‐Directed Therapy Collaborative Group. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Winters BD, Eberlein M, Leung J, Needham DM, Pronovost PJ, Sevransky JE. Long‐term mortality and quality of life in sepsis: a systematic review. Crit Care Med. 2010;38(5):1276–1283.
Cuthbertson BH, Elders A, Hall S, et al.; the Scottish Critical Care Trials Group and the Scottish Intensive Care Society Audit Group. Mortality and quality of life in the five years after severe sepsis. Crit Care. 2013;17(2):R70.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Post‐Discharge Health Care Use Is Markedly Higher in Survivors of Severe Sepsis. Am J Respir Crit Care Med 2013;187:A1573.
Perl TM, Dvorak L, Hwang T, Wenzel RP. Long‐term survival and function after suspected gram‐negative sepsis. JAMA. 1995;274(4):338–345.
Weycker D, Akhras KS, Edelsberg J, Angus DC, Oster G. Long‐term mortality and medical care charges in patients with severe sepsis. Crit Care Med. 2003;31(9):2316–2323.
Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30‐day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA. 2013;309(4):355–363.
Gwadry‐Sridhar FH, Flintoft V, Lee DS, Lee H, Guyatt GH. A systematic review and meta‐analysis of studies comparing readmission rates and mortality rates in patients with heart failure. Arch Intern Med. 2004;164(21):2315–2320.
Gheorghiade M, Braunwald E. Hospitalizations for heart failure in the United States—a sign of hope. JAMA. 2011;306(15):1705–1706.
Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688–1698.
Iwashyna TJ, Odden AJ. Sepsis after Scotland: enough with the averages, show us the effect modifiers. Crit Care. 2013;17(3):148.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11): 483–493.
Iwashyna TJ, Odden A, Rohde J, et al. Identifying patients with severe sepsis using administrative claims: patient‐level validation of the Angus implementation of the International Consensus Conference Definition of Severe Sepsis [published online ahead of print September 18, 2012]. Med Care. doi: 10.1097/MLR.0b013e318268ac86. Epub ahead of print.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2009;63(7):798–803.
Cowen ME, Dusseau DJ, Toth BG, Guisinger C, Zodet MW, Shyr Y. Casemix adjustment of managed care claims data using the clinical classification for health policy research method. Med Care. 1998;36(7):1108–1113.
Agency for Healthcare Research and Quality Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD‐9‐CM Fact Sheet. Available at: http://www.hcup‐us.ahrq.gov/toolssoftware/ccs/ccsfactsheet.jsp. Accessed January 20, 2013.
Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1997;94(446):496–509.
Braun L, Riedel AA, Cooper LM. Severe sepsis in managed care: analysis of incidence, one‐year mortality, and associated costs of care. J Manag Care Pharm. 2004;10(6):521–530.
Lee H, Doig CJ, Ghali WA, Donaldson C, Johnson D, Manns B. Detailed cost analysis of care for survivors of severe sepsis. Crit Care Med. 2004;32(4):981–985.
Rico Crescencio JC, Leu M, Balaventakesh B, Loganathan R, et al. Readmissions among patients with severe sepsis/septic shock among inner‐city minority New Yorkers. Chest. 2012;142:286A.
Czaja AS, Zimmerman JJ, Nathens AB. Readmission and late mortality after pediatric severe sepsis. Pediatrics. 2009;123(3):849–857.
Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):1418–1428.
Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100–102.
Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–1655.
Martin GS, Mannino DM, Moss M. The effect of age on the development and outcome of adult sepsis. Crit Care Med. 2006;34(1):15–21.
Pinsky MR, Matuschak GM. Multiple systems organ failure: failure of host defense homeostasis. Crit Care Clin. 1989;5(2):199–220.
Remick DG. Pathophysiology of sepsis. Am J Pathol. 2007;170(5):1435–1444.
Angus DC, Carlet J. Surviving intensive care: a report from the 2002 Brussels Roundtable. Intensive Care Med. 2003;29(3):368–377.
Siami S, Annane D, Sharshar T. The encephalopathy in sepsis. Crit Care Clin. 2008;24(1):67–82, viii.
Gofton TE, Young GB. Sepsis‐associated encephalopathy. Nat Rev Neurol. 2012;8(10):557–566.
Needham DM, Davidson J, Cohen H, et al. Improving long‐term outcomes after discharge from intensive care unit: report from a stakeholders' conference. Crit Care Med. 2012;40(2):502–509.
Liu V, Turk BJ, Rizk NW, Kipnis P, Escobar GJ. The association between sepsis and potential medical injury among hospitalized patients. Chest. 2012;142(3):606–613.
Wunsch H, Guerra C, Barnato AE, Angus DC, Li G, Linde‐Zwirble WT. Three‐year outcomes for Medicare beneficiaries who survive intensive care. JAMA. 2010;303(9):849–856.
Clermont G, Angus DC, Linde‐Zwirble WT, Griffin MF, Fine MJ, Pinsky MR. Does acute organ dysfunction predict patient‐centered outcomes? Chest. 2002;121(6):1963–1971.
Iwashyna TJ, Netzer G, Langa KM, Cigolle C. Spurious inferences about long‐term outcomes: the case of severe sepsis and geriatric conditions. Am J Respir Crit Care Med. 2012;185(8):835–841.
Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Reed M, Huang J, Graetz I, et al., Outpatient electronic health records and the clinical care and outcomes of patients with diabetes mellitus. Ann Intern Med. 2012;157(7):482–489.
Gordon NP. Similarity of the adult Kaiser Permanente membership in Northern California to the insured and general population in Northern California: statistics from the 2009 California Health Interview Survey. Internal Division of Research Report. Oakland, CA: Kaiser Permanente Division of Research; January 24, 2012. Available at: http://www.dor.kaiser.org/external/chis_non_kp_2009. Accessed January 20, 2013.
Lindenauer PK, Lagu T, Shieh MS, Pekow PS, Rothberg MB. Association of diagnostic coding with trends in hospitalizations and mortality of patients with pneumonia, 2003–2009. JAMA. 2012;307(13):1405–1413.
Sarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012;307(13):1433–1435.
Kahn JM, Rubenfeld GD, Rohrbach J, Fuchs BD. Cost savings attributable to reductions in intensive care unit length of stay for mechanically ventilated patients. Med Care. 2008;46(12):1226–1233.

References

Angus DC. The lingering consequences of sepsis: a hidden public health disaster? JAMA. 2010;304(16):1833–1834.
Dellinger RP, Levy MM, Rhodes A, et al.; Surviving Sepsis Campaign Guidelines Committee including the Pediatric Subgroup. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med. 2013;41(2):580–637.
Pfuntner A, Wier LM, Steiner C. Costs for hospital stays in the United States, 2010. HCUP statistical brief #16. January 2013. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb146.pdf. Accessed October 1, 2013.
Martin GS, Mannino DM, Eaton S, Moss M. The epidemiology of sepsis in the United States from 1979 through 2000. N Engl J Med. 2003;348(16):1546–1554.
Angus DC, Linde‐Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
Dombrovskiy VY, Martin AA, Sunderram J, Paz HL. Rapid increase in hospitalization and mortality rates for severe sepsis in the United States: a trend analysis from 1993 to 2003. Crit Care Med. 2007;35(5):1244–1250.
Elixhauser A, Friedman B, Stranges E. Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. October 2011. Rockville, MD: Agency for Healthcare Research and Quality; 2011. Available at: http://www.hcup‐us.ahrq.gov/reports/statbriefs/sb122.pdf. Accessed October 1, 2013.
Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK. Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to 2007. Crit Care Med. 2012;40(3):754–761.
Iwashyna TJ, Cooke CR, Wunsch H, Kahn JM. Population burden of long‐term survivorship after severe sepsis in older Americans. J Am Geriatr Soc. 2012;60(6):1070–1077.
Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41(5):1167–1174.
Levy MM, Artigas A, Phillips GS, et al. Outcomes of the Surviving Sepsis Campaign in intensive care units in the USA and Europe: a prospective cohort study. Lancet Infect Dis. 2012;12(12):919–924.
Townsend SR, Schorr C, Levy MM, Dellinger RP. Reducing mortality in severe sepsis: the Surviving Sepsis Campaign. Clin Chest Med. 2008;29(4):721–733, x.
Rivers E, Nguyen B, Havstad S, et al.; Early Goal‐Directed Therapy Collaborative Group. Early goal‐directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345(19):1368–1377.
Iwashyna TJ, Ely EW, Smith DM, Langa KM. Long‐term cognitive impairment and functional disability among survivors of severe sepsis. JAMA. 2010;304(16):1787–1794.
Winters BD, Eberlein M, Leung J, Needham DM, Pronovost PJ, Sevransky JE. Long‐term mortality and quality of life in sepsis: a systematic review. Crit Care Med. 2010;38(5):1276–1283.
Cuthbertson BH, Elders A, Hall S, et al.; the Scottish Critical Care Trials Group and the Scottish Intensive Care Society Audit Group. Mortality and quality of life in the five years after severe sepsis. Crit Care. 2013;17(2):R70.
Prescott HC, Langa KM, Liu V, Escobar GJ, Iwashyna TJ. Post‐Discharge Health Care Use Is Markedly Higher in Survivors of Severe Sepsis. Am J Respir Crit Care Med 2013;187:A1573.
Perl TM, Dvorak L, Hwang T, Wenzel RP. Long‐term survival and function after suspected gram‐negative sepsis. JAMA. 1995;274(4):338–345.
Weycker D, Akhras KS, Edelsberg J, Angus DC, Oster G. Long‐term mortality and medical care charges in patients with severe sepsis. Crit Care Med. 2003;31(9):2316–2323.
Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30‐day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA. 2013;309(4):355–363.
Gwadry‐Sridhar FH, Flintoft V, Lee DS, Lee H, Guyatt GH. A systematic review and meta‐analysis of studies comparing readmission rates and mortality rates in patients with heart failure. Arch Intern Med. 2004;164(21):2315–2320.
Gheorghiade M, Braunwald E. Hospitalizations for heart failure in the United States—a sign of hope. JAMA. 2011;306(15):1705–1706.
Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688–1698.
Iwashyna TJ, Odden AJ. Sepsis after Scotland: enough with the averages, show us the effect modifiers. Crit Care. 2013;17(3):148.
Whippy A, Skeath M, Crawford B, et al. Kaiser Permanente's performance improvement system, part 3: multisite improvements in care for patients with sepsis. Jt Comm J Qual Patient Saf. 2011;37(11): 483–493.
Iwashyna TJ, Odden A, Rohde J, et al. Identifying patients with severe sepsis using administrative claims: patient‐level validation of the Angus implementation of the International Consensus Conference Definition of Severe Sepsis [published online ahead of print September 18, 2012]. Med Care. doi: 10.1097/MLR.0b013e318268ac86. Epub ahead of print.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Greene JD, Scheirer P, Gardner MN, Draper D, Kipnis P. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Turk BJ, Ragins AI, Kipnis P, Escobar GJ. An electronic Simplified Acute Physiology Score‐based risk adjustment score for critical illness in an integrated healthcare system. Crit Care Med. 2013;41(1):41–48.
Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446–453.
Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2009;63(7):798–803.
Cowen ME, Dusseau DJ, Toth BG, Guisinger C, Zodet MW, Shyr Y. Casemix adjustment of managed care claims data using the clinical classification for health policy research method. Med Care. 1998;36(7):1108–1113.
Agency for Healthcare Research and Quality Healthcare Cost and Utilization Project. Clinical Classifications Software (CCS) for ICD‐9‐CM Fact Sheet. Available at: http://www.hcup‐us.ahrq.gov/toolssoftware/ccs/ccsfactsheet.jsp. Accessed January 20, 2013.
Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1997;94(446):496–509.
Braun L, Riedel AA, Cooper LM. Severe sepsis in managed care: analysis of incidence, one‐year mortality, and associated costs of care. J Manag Care Pharm. 2004;10(6):521–530.
Lee H, Doig CJ, Ghali WA, Donaldson C, Johnson D, Manns B. Detailed cost analysis of care for survivors of severe sepsis. Crit Care Med. 2004;32(4):981–985.
Rico Crescencio JC, Leu M, Balaventakesh B, Loganathan R, et al. Readmissions among patients with severe sepsis/septic shock among inner‐city minority New Yorkers. Chest. 2012;142:286A.
Czaja AS, Zimmerman JJ, Nathens AB. Readmission and late mortality after pediatric severe sepsis. Pediatrics. 2009;123(3):849–857.
Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee‐for‐service program. N Engl J Med. 2009;360(14):1418–1428.
Krumholz HM. Post‐hospital syndrome—an acquired, transient condition of generalized risk. N Engl J Med. 2013;368(2):100–102.
Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644–1655.
Martin GS, Mannino DM, Moss M. The effect of age on the development and outcome of adult sepsis. Crit Care Med. 2006;34(1):15–21.
Pinsky MR, Matuschak GM. Multiple systems organ failure: failure of host defense homeostasis. Crit Care Clin. 1989;5(2):199–220.
Remick DG. Pathophysiology of sepsis. Am J Pathol. 2007;170(5):1435–1444.
Angus DC, Carlet J. Surviving intensive care: a report from the 2002 Brussels Roundtable. Intensive Care Med. 2003;29(3):368–377.
Siami S, Annane D, Sharshar T. The encephalopathy in sepsis. Crit Care Clin. 2008;24(1):67–82, viii.
Gofton TE, Young GB. Sepsis‐associated encephalopathy. Nat Rev Neurol. 2012;8(10):557–566.
Needham DM, Davidson J, Cohen H, et al. Improving long‐term outcomes after discharge from intensive care unit: report from a stakeholders' conference. Crit Care Med. 2012;40(2):502–509.
Liu V, Turk BJ, Rizk NW, Kipnis P, Escobar GJ. The association between sepsis and potential medical injury among hospitalized patients. Chest. 2012;142(3):606–613.
Wunsch H, Guerra C, Barnato AE, Angus DC, Li G, Linde‐Zwirble WT. Three‐year outcomes for Medicare beneficiaries who survive intensive care. JAMA. 2010;303(9):849–856.
Clermont G, Angus DC, Linde‐Zwirble WT, Griffin MF, Fine MJ, Pinsky MR. Does acute organ dysfunction predict patient‐centered outcomes? Chest. 2002;121(6):1963–1971.
Iwashyna TJ, Netzer G, Langa KM, Cigolle C. Spurious inferences about long‐term outcomes: the case of severe sepsis and geriatric conditions. Am J Respir Crit Care Med. 2012;185(8):835–841.
Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Reed M, Huang J, Graetz I, et al., Outpatient electronic health records and the clinical care and outcomes of patients with diabetes mellitus. Ann Intern Med. 2012;157(7):482–489.
Gordon NP. Similarity of the adult Kaiser Permanente membership in Northern California to the insured and general population in Northern California: statistics from the 2009 California Health Interview Survey. Internal Division of Research Report. Oakland, CA: Kaiser Permanente Division of Research; January 24, 2012. Available at: http://www.dor.kaiser.org/external/chis_non_kp_2009. Accessed January 20, 2013.
Lindenauer PK, Lagu T, Shieh MS, Pekow PS, Rothberg MB. Association of diagnostic coding with trends in hospitalizations and mortality of patients with pneumonia, 2003–2009. JAMA. 2012;307(13):1405–1413.
Sarrazin MS, Rosenthal GE. Finding pure and simple truths with administrative data. JAMA. 2012;307(13):1433–1435.
Kahn JM, Rubenfeld GD, Rohrbach J, Fuchs BD. Cost savings attributable to reductions in intensive care unit length of stay for mechanically ventilated patients. Med Care. 2008;46(12):1226–1233.

Issue

Journal of Hospital Medicine - 9(8)

Issue

Journal of Hospital Medicine - 9(8)

Page Number

502-507

Page Number

502-507

Publications

Publications

Article Type

Display Headline

Hospital readmission and healthcare utilization following sepsis in community settings

Display Headline

Hospital readmission and healthcare utilization following sepsis in community settings

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Vincent Liu, MD, 2000 Broadway, Oakland, CA 94612; Telephone: 510‐627‐3621; Fax: 510‐627‐2573; E‐mail: vincent.x.liu@kp.org

Content Gating

No Gating (article Unlocked/Free)

Alternative CME

Teaser Media

Article PDF Media

Media Files

Electronic Order Set for AMI

Article Type

Changed

Sun, 05/21/2017 - 14:39

Display Headline

An electronic order set for acute myocardial infarction is associated with improved patient outcomes through better adherence to clinical practice guidelines

Author(s)

Although the prevalence of coronary heart disease and death from acute myocardial infarction (AMI) have declined steadily, about 935,000 heart attacks still occur annually in the United States, with approximately one‐third of these being fatal.[1, 2, 3] Studies have demonstrated decreased 30‐day and longer‐term mortality in AMI patients who receive evidence‐based treatment, including aspirin, ‐blockers, angiotensin‐converting enzyme (ACE) inhibitors or angiotensin receptor blockers (ARBs), anticoagulation therapy, and statins.[4, 5, 6, 7] Despite clinical practice guidelines (CPGs) outlining evidence‐based care and considerable efforts to implement processes that improve patient outcomes, delivery of effective therapy remains suboptimal.[8] For example, the Hospital Quality Alliance Program[9] found that in AMI patients, use of aspirin on admission was only 81% to 92%, ‐blocker on admission 75% to 85%, and ACE inhibitors for left ventricular dysfunction 71% to 74%.

Efforts to increase adherence to CPGs and improve patient outcomes in AMI have resulted in variable degrees of success. They include promotion of CPGs,[4, 5, 6, 7] physician education with feedback, report cards, care paths, registries,[10] Joint Commission standardized measures,[11] and paper checklists or order sets (OS).[12, 13]

In this report, we describe the association between use of an evidence‐based, electronic OS for AMI (AMI‐OS) and better adherence to CPGs. This AMI‐OS was implemented in the inpatient electronic medical records (EMRs) of a large integrated healthcare delivery system, Kaiser Permanente Northern California (KPNC). The purpose of our investigation was to determine (1) whether use of the AMI‐OS was associated with improved AMI processes and patient outcomes, and (2) whether these associations persisted after risk adjustment using a comprehensive severity of illness scoring system.

MATERIALS AND METHODS

This project was approved by the KPNC institutional review board.

Under a mutual exclusivity arrangement, salaried physicians of The Permanente Medical Group, Inc., care for 3.4 million Kaiser Foundation Health Plan, Inc. members at facilities owned by Kaiser Foundation Hospitals, Inc. All KPNC facilities employ the same information systems with a common medical record number and can track care covered by the plan but delivered elsewhere.[14] Our setting consisted of 21 KPNC hospitals described in previous reports,[15, 16, 17, 18] using the same commercially available EMR system that includes computerized physician order entry (CPOE). Deployment of the customized inpatient Epic EMR (www.epicsystems.com), known internally as KP HealthConnect (KPHC), began in 2006 and was completed in 2010.

In this EMR's CPOE, physicians have options to select individual orders (a la carte) or they can utilize an OS, which is a collection of the most appropriate orders associated with specific diagnoses, procedures, or treatments. The evidence‐based AMI‐OS studied in this project was developed by a multidisciplinary team (for detailed components see Supporting Appendix 1Appendix 5 in the online version of this article).

Our study focused on the first set of hospital admission orders for patients with AMI. The study sample consisted of patients meeting these criteria: (1) age 18 years at admission; (2) admitted to a KPNC hospital for an overnight stay between September 28, 2008 and December 31, 2010; (3) principal diagnosis was AMI (International Classification of Diseases, 9th Revision [ICD‐9][19] codes 410.00, 01, 10, 11, 20, 21, 30, 31, 40, 41, 50, 51, 60, 61, 70, 71, 80, 90, and 91); and (4) KPHC had been operational at the hospital for at least 3 months to be included (for assembly descriptions see Supporting Appendices 15 in the online version of this article). At the study hospitals, troponin I was measured using the Beckman Access AccuTnI assay (Beckman Coulter, Inc., Brea, CA), whose upper reference limit (99th percentile) is 0.04 ng/mL. We excluded patients initially hospitalized for AMI at a non‐KPNC site and transferred into a study hospital.

The data processing methods we employed have been detailed elsewhere.[14, 15, 17, 20, 21, 22] The dependent outcome variables were total hospital length of stay, inpatient mortality, 30‐day mortality, and all‐cause rehospitalization within 30 days of discharge. Linked state mortality data were unavailable for the entire study period, so we ascertained 30‐day mortality based on the combination of KPNC patient demographic data and publicly available Social Security Administration decedent files. We ascertained rehospitalization by scanning KPNC hospitalization databases, which also track out‐of‐plan use.

The dependent process variables were use of aspirin within 24 hours of admission, ‐blockers, anticoagulation, ACE inhibitors or ARBs, and statins. The primary independent variable of interest was whether or not the admitting physician employed the AMI‐OS when admission orders were entered. Consequently, this variable is dichotomous (AMI‐OS vs a la carte).

We controlled for acute illness severity and chronic illness burden using a recent modification[22] of an externally validated risk‐adjustment system applicable to all hospitalized patients.[15, 16, 23, 24, 25] Our methodology included vital signs, neurological status checks, and laboratory test results obtained in the 72 hours preceding hospital admission; comorbidities were captured longitudinally using data from the year preceding hospitalization (for comparison purposes, we also assigned a Charlson Comorbidity Index score[26]).

End‐of‐life care directives are mandatory on admission at KPNC hospitals. Physicians have 4 options: full code, partial code, do not resuscitate, and comfort care only. Because of small numbers in some categories, we collapsed these 4 categories into full code and not full code. Because patients' care directives may change, we elected to capture the care directive in effect when a patient first entered a hospital unit other than the emergency department (ED).

Two authors (M.B., P.C.L.), one of whom is a board‐certified cardiologist, reviewed all admission electrocardiograms and made a consensus determination as to whether or not criteria for ST‐segment elevation myocardial infarction (STEMI) were present (ie, new ST‐segment elevation or left bundle branch block); we also reviewed the records of all patients with missing troponin I data to confirm the AMI diagnosis.

Statistical Methods

We performed unadjusted comparisons between AMI‐OS and nonAMI‐OS patients using the t test or the [2] test, as appropriate.

We hypothesized that the AMI‐OS plays a mediating role on patient outcomes through its effect on adherence to recommended treatment. We evaluated this hypothesis for inpatient mortality by first fitting a multivariable logistic regression model for inpatient mortality as the outcome and either the 5 evidence‐based therapies or the total number of evidence‐based therapies used (ranging from 02, 3, 4, or 5) as the dependent variable controlling for age, gender, presence of STEMI, troponin I, comorbidities, illness severity, ED length of stay (LOS), care directive status, and timing of cardiac catheterization referral as covariates to confirm the protective effect of these therapies on mortality. We then used the same model to estimate the effect of AMI‐OS on inpatient mortality, substituting the therapies with AMI‐OS as the dependent variable and using the same covariates. Last, we included both the therapies and the AMI‐OS in the model to evaluate their combined effects.[27]

We used 2 different methods to estimate the effects of AMI‐OS and number of therapies provided on the outcomes while adjusting for observed baseline differences between the 2 groups of patients: propensity risk score matching, which estimates the average treatment effect for the treated,[28, 29] and inverse probability of treatment weighting, which is used to estimate the average treatment effect.[30, 31, 32] The propensity score was defined as the probability of receiving the intervention for a patient with specific predictive factors.[33, 34] We computed a propensity score for each patient by using logistic regression, with the dependent variable being receipt of AMI‐OS and the independent variables being the covariates used for the multivariate logistic regression as well as ICD‐9 code for final diagnosis. We calculated the Mahalanobis distance between patients who received AMI‐OS (cases) and patients who did not received AMI‐OS (controls) using the same set of covariates. We matched each case to a single control within the same facility based on the nearest available Mahalanobis metric matching within calipers defied as the maximum width of 0.2 standard deviations of the logit of the estimated propensity score.[29, 35] We estimated the odds ratios for the binary dependent variables based on a conditional logistic regression model to account for the matched pairs design.[28] We used a generalized linear model with the log‐transformed LOS as the outcome to estimate the ratio of the LOS geometric mean of the cases to the controls. We calculated the relative risk for patients receiving AMI‐OS via the inverse probability weighting method by first defining a weight for each patient. [We assigned a weight of 1/ps_i to patients who received the AMI‐OS and a weight of 1/(1ps_i) to patients who did not receive the AMI‐OS, where ps_i denotes the propensity score for patient i]. We used a logistic regression model for the binary dependent variables with the same set of covariates described above to estimate the adjusted odds ratios while weighting each observation by its corresponding weight. Last, we used a weighted generalized linear model to estimate the AMI‐OS effect on the log‐transformed LOS.

RESULTS

Table 1 summarizes the characteristics of the 5879 patients. It shows that AMI‐OS patients were more likely to receive evidence‐based therapies for AMI (aspirin, ‐blockers, ACE inhibitors or ARBs, anticoagulation, and statins) and had a 46% lower mortality rate in hospital (3.51 % vs 6.52%) and 33% lower rate at 30 days (5.66% vs 8.48%). AMI‐OS patients were also found to be at lower risk for an adverse outcome than nonAMI‐OS patients. The AMI‐OS patients had lower peak troponin I values, severity of illness (lower Laboratory‐Based Acute Physiology Score, version 2 [LAPS2] scores), comorbidity burdens (lower Comorbidity Point Score, version 2 [COPS2] and Charlson scores), and global predicted mortality risk. AMI‐OS patients were also less likely to have required intensive care. AMI‐OS patients were at higher risk of death than nonAMI‐OS patients with respect to only 1 variable (being full code at the time of admission), but although this difference was statistically significant, it was of minor clinical impact (86% vs 88%).

Table 1.Description of Study Cohort
	Patients Initially Managed Using		P Valuea
	AMI Order Set, N=3,531b	A La Carte Orders, N=2,348b	P Valuea
NOTE: Abbreviations: ACE, angiotensin‐converting enzyme; AMI, acute myocardial infarction; AMI‐OS, acute myocardial infarction order set; ARBs, angiotensin receptor blockers; COPS2, Comorbidity Point Score, version 2; CPOE, computerized physician order entry; ED, emergency department; ICU, intensive care unit; LAPS2, Laboratory‐based Acute Physiology Score, version 2; SD, standard deviation; STEMI, ST‐segment elevation myocardial infarction. a ² or t test, as appropriate. See text for further methodological details. b AMI‐OS is an evidence‐based electronic checklist that guides physicians to order the most effective therapy by CPOE during the hospital admission process. In contrast, a la carte means that the clinician did not use the AMI‐OS, but rather entered individual orders via CPOE. See text for further details. c STEMI as evident by electrocardiogram. See text for details on ascertainment. d See text and reference 31 for details on how this score was assigned. e The COPS2 is a longitudinal, diagnosis‐based score assigned monthly that integrates all diagnoses incurred by a patient in the preceding 12 months. It is a continuous variable that can range between a minimum of zero and a theoretical maximum of 1,014, although <0.05% of Kaiser Permanente hospitalized patients have a COPS2 exceeding 241, and none have had a COPS2 >306. Increasing values of the COPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the COPS2. The LAPS2 integrates results from vital signs, neurological status checks, and 15 laboratory tests in the 72 hours preceding hospitalization into a single continuous variable. Increasing degrees of physiologic derangement are reflected in a higher LAPS2, which can range between a minimum of zero and a theoretical maximum of 414, although <0.05% of Kaiser Permanente hospitalized patients have a LAPS2 exceeding 227, and none have had a LAPS2 >282. Increasing values of LAPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the LAPS2. f See text for details of specific therapies and how they were ascertained using the electronic medical record. g Percent mortality risk based on age, sex, diagnosis, COPS2, LAPS2, and care directive using a predictive model described in text and in reference 22. h See text for description of how end‐of‐life care directives are captured in the electronic medical record. i Direct admit means that the first hospital unit in which a patient stayed was the ICU; transfer refers to those patients transferred to the ICU from another unit in the hospital.
Age, y, median (meanSD)	70 (69.413.8)	70 (69.213.8)	0.5603
Age (% >65 years)	2,134 (60.4%)	1,415 (60.3%)	0.8949
Sex (% male)	2,202 (62.4%)	1,451 (61.8%)	0.6620
STEMI (% with)c	166 (4.7%)	369 (15.7%)	<0.0001
Troponin I (% missing)	111 (3.1%)	151 (6.4%)	<0.0001
Troponin I median (meanSD)	0.57 (3.08.2)	0.27 (2.58.9)	0.0651
Charlson score median (meanSD)d	2.0 (2.51.5)	2.0 (2.71.6)	<0.0001
COPS2, median (meanSD)e	14.0 (29.831.7)	17.0 (34.334.4)	<0.0001
LAPS2, median (meanSD)e	0.0 (35.643.5)	27.0 (40.948.1)	<0.0001
Length of stay in ED, h, median (meanSD)	5.7 (5.93.0)	5.7 (5.43.1)	<0.0001
Patients receiving aspirin within 24 hoursf	3,470 (98.3%)	2,202 (93.8%)	<0.0001
Patients receiving anticoagulation therapyf	2,886 (81.7%)	1,846 (78.6%)	0.0032
Patients receiving ‐blockersf	3,196 (90.5%)	1,926 (82.0%)	<0.0001
Patients receiving ACE inhibitors or ARBsf	2,395 (67.8%)	1,244 (53.0%)	<0.0001
Patients receiving statinsf	3,337 (94.5%)	1,975 (84.1%)	<0.0001
Patient received 1 or more therapies	3,531 (100.0%)	2,330 (99.2%)	<0.0001
Patient received 2 or more therapies	3,521 (99.7%)	2,266 (96.5%)	<0.0001
Patient received 3 or more therapies	3,440 (97.4%)	2,085 (88.8%)	<0.0001
Patient received 4 or more therapies	3,015 (85.4%)	1,646 (70.1%)	<0.0001
Patient received all 5 therapies	1,777 (50.3%)	866 (35.9%)	<0.0001
Predicted mortality risk, %, median, (meanSD)f	0.86 (3.27.4)	1.19 (4.810.8)	<0.0001
Full code at time of hospital entry (%)g	3,041 (86.1%)	2,066 (88.0%)	0.0379
Admitted to ICU (%)i
Direct admit	826 (23.4%)	567 (24.2%)	0.5047
Unplanned transfer	222 (6.3%)	133 (5.7%)	0.3262
Ever	1,283 (36.3%)	1,169 (49.8%)	<0.0001
Length of stay, h, median (meanSD)	68.3 (109.4140.9)	68.9 (113.8154.3)	0.2615
Inpatient mortality (%)	124 (3.5%)	153 (6.5%)	<0.0001
30‐day mortality (%)	200 (5.7%)	199 (8.5%)	<0.0001
All‐cause rehospitalization within 30 days (%)	576 (16.3%)	401 (17.1%)	0.4398
Cardiac catheterization procedure referral timing
1 day preadmission to discharge	2,018 (57.2%)	1,348 (57.4%)	0.1638
2 days preadmission or earlier	97 (2.8%)	87 (3.7%)
After discharge	149 (4.2%)	104 (4.4%)
No referral	1,267 (35.9%)	809 (34.5%)

Table 2 shows the result of a logistic regression model in which the dependent variable was inpatient mortality and either the 5 evidence‐based therapies or the total number of evidence‐based therapies are the dependent variables. ‐blocker, statin, and ACE inhibitor or ARB therapies all had a protective effect on mortality, with odds ratios ranging from 0.48 (95% confidence interval [CI]: 0.36‐0.64), 0.63 (95% CI: 0.45‐0.89), and 0.40 (95% CI: 0.30‐0.53), respectively. An increased number of therapies also had a beneficial effect on inpatient mortality, with patients having 3 or more of the evidence‐based therapies showing an adjusted odds ratio (AOR) of 0.49 (95% CI: 0.33‐0.73), 4 or more therapies an AOR of 0.29 (95% CI: 0.20‐0.42), and 0.17 (95% CI: 0.11‐0.25) for 5 or more therapies.

Table 2.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Evidence‐Based Therapies
	Multiple Therapies Effect		Individual Therapies Effect
Outcome	Death		Death
Number of outcomes	277		277
	AORa	95% CIb	AORa	95% CIb
NOTE: Abbreviations: ACE = angiotensin converting enzyme; ARB = angiotensin receptor blockers. a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized.
Age in years
1839	Ref		Ref
4064	1.02	(0.147.73)	1.01	(0.137.66)
6584	4.05	(0.5529.72)	3.89	(0.5328.66)
85+	4.99	(0.6737.13)	4.80	(0.6435.84)
Sex
Female	Ref
Male	1.05	(0.811.37)	1.07	(0.821.39)
STEMIc
Absent	Ref		Ref
Present	4.00	(2.755.81)	3.86	(2.645.63)
Troponin I
0.1 ng/ml	Ref		Ref
>0.1 ng/ml	1.01	(0.721.42)	1.02	(0.731.43)
COPS2d (AOR per 10 points)	1.05	(1.011.08)	1.04	(1.011.08)
LAPS2d (AOR per 10 points)	1.09	(1.061.11)	1.09	(1.061.11)
ED LOSe (hours)
<6	Ref		Ref
67	0.74	(0.531.03)	0.76	(0.541.06)
>=12	0.82	(0.391.74)	0.83	(0.391.78)
Code Statusf
Full Code	Ref
Not Full Code	1.08	(0.781.49)	1.09	(0.791.51)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.40	(0.290.54)	0.39	(0.280.53)
Number of therapies received
2 or less	Ref
3	0.49	(0.330.73)
4	0.29	(0.200.42)
5	0.17	(0.110.25)
Aspirin therapy			0.80	(0.491.32)
Anticoagulation therapy			0.86	(0.641.16)
Beta Blocker therapy			0.48	(0.360.64)
Statin therapy			0.63	(0.450.89)
ACE inhibitors or ARBs			0.40	(0.300.53)
C Statistic	0.814		0.822
Hosmer‐Lemeshow p value	0.509		0.934

Table 3 shows that the use of the AMI‐OS is protective, with an AOR of 0.59 and a 95% CI of 0.45‐0.76. Table 3 also shows that the most potent predictors were comorbidity burden (AOR: 1.07, 95% CI: 1.03‐1.10 per 10 COPS2 points), severity of illness (AOR: 1.09, 95% CI: 1.07‐1.12 per 10 LAPS2 points), STEMI (AOR: 3.86, 95% CI: 2.68‐5.58), and timing of cardiac catheterization referral occurring immediately prior to or during the admission (AOR: 0.37, 95% CI: 0.27‐0.51). The statistical significance of the AMI‐OS effect disappears when both AMI‐OS and the individual therapies are included in the same model (see Supporting Information, Appendices 15, in the online version of this article).

Table 3.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Acute Myocardial Infarction Order Set
Outcome	Death
Number of outcomes	277
	AORa	95% CIb
a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized. g ^**See text for details on the order set.
Age in years
1839	Ref
4064	1.16	(0.158.78)
6584	4.67	(0.6334.46)
85+	5.45	(0.7340.86)
Sex
Female	Ref
Male	1.05	(0.811.36)
STEMIc
Absent	Ref
Present	3.86	(2.685.58)
Troponin I
0.1 ng/ml	Ref
>0.1 ng/ml	1.16	(0.831.62)
COPS2d (AOR per 10 points)	1.07	(1.031.10)
LAPS2d (AOR per 10 points)	1.09	(1.071.12)
ED LOSe (hours)
<6	Ref
67	0.72	(0.521.00)
>=12	0.70	(0.331.48)
Code statusf
Full code	Ref
Not full code	1.22	(0.891.68)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.37	(0.270.51)
Order set employedg
No	Ref
Yes	0.59	(0.450.76)
C Statistic	0.792
Hosmer‐Lemeshow p value	0.273

Table 4 shows separately the average treatment effect (ATE) and average treatment effect for the treated (ATT) of AMI‐OS and of increasing number of therapies on other outcomes (30‐day mortality, LOS, and readmission). Both the ATE and ATT show that the use of the AMI‐OS was significantly protective with respect to mortality and total hospital LOS but not significant with respect to readmission. The effect of the number of therapies on mortality is significantly higher with increasing number of therapies. For example, patients who received 5 therapies had an average treatment effect on 30‐day inpatient mortality of 0.23 (95% CI: 0.15‐0.35) compared to 0.64 (95% CI: 0.43‐0.96) for 3 therapies, almost a 3‐fold difference. The effects of increasing number of therapies were not significant for LOS or readmission. A sensitivity analysis in which the 535 STEMI patients were removed showed essentially the same results, so it is not reported here.

Table 4.Adjusted Odds Ratio (95% CI) or Mean Length‐of‐Stay Ratio (95% CI) in Study Patients
Outcome	Order Seta	3 Therapiesb	4 Therapiesb	5 Therapiesb
NOTE: Abbreviations: CI, confidence interval; LOS, length of stay. a Refers to comparison in which the reference group consists of patients who were not treated using the acute myocardial infarction order set. b Refers to comparison in which the reference group consists of patients who received 2 or less of the 5 recommended therapies. c See text for description of average treatment effect methodology. d See text for description of average treatment effect on the treated and matched pair adjustment methodology. e See text for details on how we modeled LOS.
Average treatment effectc
Inpatient mortality	0.67 (0.520.86)	0.64 (0.430.96)	0.37 (0.250.54)	0.23 (0.150.35)
30‐day mortality	0.77 (0.620.96)	0.68 (0.480.98)	0.34 (0.240.48)	0.26 (0.180.37)
Readmission	1.03 (0.901.19)	1.20 (0.871.66)	1.19 (0.881.60)	1.30 (0.961.76)
LOS, ratio of the geometric means	0.91 (0.870.95)	1.16 (1.031.30)	1.17 (1.051.30)	1.12 (1.001.24)
Average treatment effect on the treatedd
Inpatient mortality	0.69 (0.520.92)	0.35 (0.130.93)	0.17 (0.070.43)	0.08 (0.030.20)
30‐day mortality	0.84 (0.661.06)	0.35 (0.150.79)	0.17 (0.070.37)	0.09 (0.040.20)
Readmission	1.02 (0.871.20)	1.39 (0.852.26)	1.36 (0.882.12)	1.23 (0.801.89)
LOS, ratio of the geometric meanse	0.92 (0.870.97)	1.18 (1.021.37)	1.16 (1.011.33)	1.04 (0.911.19)

To further elucidate possible reasons why physicians did not use the AMI‐OS, the lead author reviewed 105 randomly selected records where the AMI‐OS was not used, 5 records from each of the 21 study hospitals. This review found that in 36% of patients, the AMI‐OS was not used because emergent catheterization or transfer to a facility with percutaneous coronary intervention capability occurred. Presence of other significant medical conditions, including critical illness, was the reason in 17% of these cases, patient or family refusal of treatments in 8%, issues around end‐of‐life care in 3%, and specific medical contraindications in 1%. In the remaining 34%, no reason for not using the AMI‐OS could be identified.

DISCUSSION

We evaluated the use of an evidence‐based electronic AMI‐OS embedded in a comprehensive EMR and found that it was beneficial. Its use was associated with increased adherence to evidence‐based therapies, which in turn were associated with improved outcomes. Using data from a large cohort of hospitalized AMI patients in 21 community hospitals, we were able to use risk adjustment that included physiologic illness severity to adjust for baseline mortality risk. Patients in whom the AMI‐OS was employed tended to be at lower risk; nonetheless, after controlling for confounding variables and adjusting for bias using propensity scores, the AMI‐OS was associated with increased use of evidence‐based therapies and decreased mortality. Most importantly, it appears that the benefits of the OS were not just due to increased receipt of individual recommended therapies, but to increased concurrent receipt of multiple recommended therapies.

Modern EMRs have great potential for significant improvements in the quality, efficiency, and safety of care provided,[36] and our study highlights this potential. However, a number of important limitations to our study must be considered. Although we had access to a very rich dataset, we could not control for all possible confounders, and our risk adjustment cannot match the level of information available to clinicians. In particular, the measurements available to us with respect to cardiac risk are limited. Thus, we have to recognize that the strength of our findings does not approximate that of a randomized trial, and one would expect that the magnitude of the beneficial association would fall under more controlled conditions. Resource limitations also did not permit us to gather more time course data (eg, sequential measurements of patient instability, cardiac damage, or use of recommended therapies), which could provide a better delineation of differences in both processes and outcomes.

Limitations also exist to the generalizability of the use of order sets in other settings that go beyond the availability of a comprehensive EMR. Our study population was cared for in a setting with an unusually high level of integration.[1] For example, KPNC has an elaborate administrative infrastructure for training in the use of the EMR as well as ensuring that order sets are not just evidence‐based, but that they are perceived by clinicians to be of significant value. This infrastructure, established to ensure physician buy‐in, may not be easy to replicate in smaller or less‐integrated settings. Thus, it is conceivable that factors other than the degree of support during the EMR deployments can affect rates of order set use.

Although our use of counterfactual methods included illness severity (LAPS2) and longitudinal comorbidity burden (COPS2), which are not yet available outside highly integrated delivery services employing comprehensive EMRs, it is possible they are insufficient. We cannot exclude the possibility that other biases or patient characteristics were present that led clinicians to preferentially employ the electronic order set in some patients but not in others. One could also argue that future studies should consider using overall adherence to recommended AMI treatment guidelines as a risk adjustment tool that would permit one to analyze what other factors may be playing a role in residual differences in patient outcomes. Last, one could object to our inclusion of STEMI patients; however, this was not a study on optimum treatment strategies for STEMI patients. Rather, it was a study on the impact on AMI outcomes of a specific component of computerized order entry outside the research setting.

Despite these limitations, we believe that our findings provide strong support for the continued use of electronic evidence‐based order sets in the inpatient medical setting. Once the initial implementation of a comprehensive EMR has occurred, deployment of these electronic order sets is a relatively inexpensive but effective method to foster compliance with evidence‐based care.

Future research in healthcare information technology can take a number of directions. One important area, of course, revolves around ways to promote enhanced physician adoption of EMRs. Our audit of records where the AMI‐OS was not used found that specific reasons for not using the order set (eg, treatment refusals, emergent intervention) were present in two‐thirds of the cases. This suggests that future analyses of adherence involving EMRs and CPOE implementation should take a more nuanced look at how order entry is actually enabled. It may be that understanding how order sets affect care enhances clinician acceptance and thus could serve as an incentive to EMR adoption. However, once an EMR is adopted, a need exists to continue evaluations such as this because, ultimately, the gold standard should be improved patient care processes and better outcomes for patients.

Acknowledgement

The authors give special thanks to Dr. Brian Hoberman for sponsoring this work, Dr. Alan S. Go for providing assistance with obtaining copies of electrocardiograms for review, Drs. Tracy Lieu and Vincent Liu for reviewing the manuscript, and Ms. Rachel Lesser for formatting the manuscript.

Disclosures: This work was supported by The Permanente Medical Group, Inc. and Kaiser Foundation Hospitals, Inc. The algorithms used to extract data and perform risk adjustment were developed with funding from the Sidney Garfield Memorial Fund (Early Detection of Impending Physiologic Deterioration in Hospitalized Patients, 1159518), the Agency for Healthcare Quality and Research (Rapid Clinical Snapshots From the EMR Among Pneumonia Patients, 1R01HS018480‐01), and the Gordon and Betty Moore Foundation (Early Detection of Impending Physiologic Deterioration: Electronic Early Warning System).

Files

References

Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Rosamond WD, Chambless LE, Heiss G, et al. Twenty‐two‐year trends in incidence of myocardial infarction, coronary heart disease mortality, and case fatality in 4 US communities, 1987–2008. Circulation. 2012;125(15):1848–1857.
Roger VL, Go AS, Lloyd‐Jones DM, et al. Heart disease and stroke statistics—2012 update: a report from the American Heart Association. Circulation. 2012;125(1):e2–e220.
Anderson JL, Adams CD, Antman EM, et al. ACC/AHA 2007 guidelines for the management of patients with unstable angina/non‐ST‐Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non‐ST‐Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007;50(7):e1–e157.
Antman EM, Hand M, Armstrong PW, et al. 2007 focused update of the ACC/AHA 2004 guidelines for the management of patients with ST‐elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2008;51(2):210–247.
Jernberg T, Johanson P, Held C, Svennblad B, Lindback J, Wallentin L. Association between adoption of evidence‐based treatment and survival for patients with ST‐elevation myocardial infarction. JAMA. 2011;305(16):1677–1684.
Puymirat E, Simon T, Steg PG, et al. Association of changes in clinical characteristics and management with improvement in survival among patients with ST‐elevation myocardial infarction. JAMA. 2012;308(10):998–1006.
Motivala AA, Cannon CP, Srinivas VS, et al. Changes in myocardial infarction guideline adherence as a function of patient risk: an end to paradoxical care? J Am Coll Cardiol. 2011;58(17):1760–1765.
Jha AK, Li Z, Orav EJ, Epstein AM. Care in U.S. hospitals—the Hospital Quality Alliance program. N Engl J Med. 2005;353(3):265–274.
Desai N, Chen AN, et al. Challenges in the treatment of NSTEMI patients at high risk for both ischemic and bleeding events: insights from the ACTION Registry‐GWTG. J Am Coll Cardiol. 2011;57:E913–E913.
Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
Eagle KA, Montoye K, Riba AL. Guideline‐based standardized care is associated with substantially lower mortality in medicare patients with acute myocardial infarction. J Am Coll Cardiol. 2005;46(7):1242–1248.
Ballard DJ, Ogola G, Fleming NS, et al. Impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care. Int J Qual Health Care. 2010;22(6):437–444.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48(8):739–744.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
International Classification of Diseases, 9th Revision‐Clinical Modification. 4th ed. 3 Vols. Los Angeles, CA: Practice Management Information Corporation; 2006.
Go AS, Hylek EM, Chang Y, et al. Anticoagulation therapy for stroke prevention in atrial fibrillation: how well do randomized trials translate into clinical practice? JAMA. 2003;290(20):2685–2692.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, David D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Kipnis P, Escobar GJ, Draper D. Effect of choice of estimation method on inter‐hospital mortality rate comparisons. Med Care. 2010;48(5):456–485.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63(7):798–803.
Wong J, Taljaard M, Forster AJ, Escobar GJ, Walraven C. Derivation and validation of a model to predict daily risk of death in hospital. Med Care. 2011;49(8):734–743.
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
MacKinnon DP. Introduction to Statistical Mediation Analysis. New York, NY: Lawrence Erlbaum Associates; 2008.
Imbens GW. Nonparametric estimation of average treatment effects under exogenity: a review. Rev Econ Stat. 2004;86:25.
Rosenbaum PR. Design of Observational Studies. New York, NY: Springer Science+Business Media; 2010.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Stat Med. 2009;28:24.
Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994(89):846–866.
Lunceford JK, Davidian M. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med. 2004;23(19):2937–2960.
Rosenbaum PR. Discussing hidden bias in observational studies. Ann Intern Med. 1991;115(11):901–905.
D'Agostino RB. Propensity score methods for bias reduction in the comparison of a treatment to a non‐randomized control group. Stat Med. 1998;17(19):2265–2281.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study, 2005. www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed on September 14, 2013.
Ettinger WH. Using health information technology to improve health care. Arch Intern Med. 2012;172(22):1728–1730.

Article PDF

Issue

Journal of Hospital Medicine - 9(3)

Publications

Page Number

155-161

Read more about Electronic Order Set for AMI

Sections

Author(s)

Author(s)

Files

Files

Article PDF

Article PDF

MATERIALS AND METHODS

This project was approved by the KPNC institutional review board.

Statistical Methods

We performed unadjusted comparisons between AMI‐OS and nonAMI‐OS patients using the t test or the [2] test, as appropriate.

RESULTS

Table 1.Description of Study Cohort
	Patients Initially Managed Using		P Valuea
	AMI Order Set, N=3,531b	A La Carte Orders, N=2,348b	P Valuea
NOTE: Abbreviations: ACE, angiotensin‐converting enzyme; AMI, acute myocardial infarction; AMI‐OS, acute myocardial infarction order set; ARBs, angiotensin receptor blockers; COPS2, Comorbidity Point Score, version 2; CPOE, computerized physician order entry; ED, emergency department; ICU, intensive care unit; LAPS2, Laboratory‐based Acute Physiology Score, version 2; SD, standard deviation; STEMI, ST‐segment elevation myocardial infarction. a ² or t test, as appropriate. See text for further methodological details. b AMI‐OS is an evidence‐based electronic checklist that guides physicians to order the most effective therapy by CPOE during the hospital admission process. In contrast, a la carte means that the clinician did not use the AMI‐OS, but rather entered individual orders via CPOE. See text for further details. c STEMI as evident by electrocardiogram. See text for details on ascertainment. d See text and reference 31 for details on how this score was assigned. e The COPS2 is a longitudinal, diagnosis‐based score assigned monthly that integrates all diagnoses incurred by a patient in the preceding 12 months. It is a continuous variable that can range between a minimum of zero and a theoretical maximum of 1,014, although <0.05% of Kaiser Permanente hospitalized patients have a COPS2 exceeding 241, and none have had a COPS2 >306. Increasing values of the COPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the COPS2. The LAPS2 integrates results from vital signs, neurological status checks, and 15 laboratory tests in the 72 hours preceding hospitalization into a single continuous variable. Increasing degrees of physiologic derangement are reflected in a higher LAPS2, which can range between a minimum of zero and a theoretical maximum of 414, although <0.05% of Kaiser Permanente hospitalized patients have a LAPS2 exceeding 227, and none have had a LAPS2 >282. Increasing values of LAPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the LAPS2. f See text for details of specific therapies and how they were ascertained using the electronic medical record. g Percent mortality risk based on age, sex, diagnosis, COPS2, LAPS2, and care directive using a predictive model described in text and in reference 22. h See text for description of how end‐of‐life care directives are captured in the electronic medical record. i Direct admit means that the first hospital unit in which a patient stayed was the ICU; transfer refers to those patients transferred to the ICU from another unit in the hospital.
Age, y, median (meanSD)	70 (69.413.8)	70 (69.213.8)	0.5603
Age (% >65 years)	2,134 (60.4%)	1,415 (60.3%)	0.8949
Sex (% male)	2,202 (62.4%)	1,451 (61.8%)	0.6620
STEMI (% with)c	166 (4.7%)	369 (15.7%)	<0.0001
Troponin I (% missing)	111 (3.1%)	151 (6.4%)	<0.0001
Troponin I median (meanSD)	0.57 (3.08.2)	0.27 (2.58.9)	0.0651
Charlson score median (meanSD)d	2.0 (2.51.5)	2.0 (2.71.6)	<0.0001
COPS2, median (meanSD)e	14.0 (29.831.7)	17.0 (34.334.4)	<0.0001
LAPS2, median (meanSD)e	0.0 (35.643.5)	27.0 (40.948.1)	<0.0001
Length of stay in ED, h, median (meanSD)	5.7 (5.93.0)	5.7 (5.43.1)	<0.0001
Patients receiving aspirin within 24 hoursf	3,470 (98.3%)	2,202 (93.8%)	<0.0001
Patients receiving anticoagulation therapyf	2,886 (81.7%)	1,846 (78.6%)	0.0032
Patients receiving ‐blockersf	3,196 (90.5%)	1,926 (82.0%)	<0.0001
Patients receiving ACE inhibitors or ARBsf	2,395 (67.8%)	1,244 (53.0%)	<0.0001
Patients receiving statinsf	3,337 (94.5%)	1,975 (84.1%)	<0.0001
Patient received 1 or more therapies	3,531 (100.0%)	2,330 (99.2%)	<0.0001
Patient received 2 or more therapies	3,521 (99.7%)	2,266 (96.5%)	<0.0001
Patient received 3 or more therapies	3,440 (97.4%)	2,085 (88.8%)	<0.0001
Patient received 4 or more therapies	3,015 (85.4%)	1,646 (70.1%)	<0.0001
Patient received all 5 therapies	1,777 (50.3%)	866 (35.9%)	<0.0001
Predicted mortality risk, %, median, (meanSD)f	0.86 (3.27.4)	1.19 (4.810.8)	<0.0001
Full code at time of hospital entry (%)g	3,041 (86.1%)	2,066 (88.0%)	0.0379
Admitted to ICU (%)i
Direct admit	826 (23.4%)	567 (24.2%)	0.5047
Unplanned transfer	222 (6.3%)	133 (5.7%)	0.3262
Ever	1,283 (36.3%)	1,169 (49.8%)	<0.0001
Length of stay, h, median (meanSD)	68.3 (109.4140.9)	68.9 (113.8154.3)	0.2615
Inpatient mortality (%)	124 (3.5%)	153 (6.5%)	<0.0001
30‐day mortality (%)	200 (5.7%)	199 (8.5%)	<0.0001
All‐cause rehospitalization within 30 days (%)	576 (16.3%)	401 (17.1%)	0.4398
Cardiac catheterization procedure referral timing
1 day preadmission to discharge	2,018 (57.2%)	1,348 (57.4%)	0.1638
2 days preadmission or earlier	97 (2.8%)	87 (3.7%)
After discharge	149 (4.2%)	104 (4.4%)
No referral	1,267 (35.9%)	809 (34.5%)

Table 2.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Evidence‐Based Therapies
	Multiple Therapies Effect		Individual Therapies Effect
Outcome	Death		Death
Number of outcomes	277		277
	AORa	95% CIb	AORa	95% CIb
NOTE: Abbreviations: ACE = angiotensin converting enzyme; ARB = angiotensin receptor blockers. a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized.
Age in years
1839	Ref		Ref
4064	1.02	(0.147.73)	1.01	(0.137.66)
6584	4.05	(0.5529.72)	3.89	(0.5328.66)
85+	4.99	(0.6737.13)	4.80	(0.6435.84)
Sex
Female	Ref
Male	1.05	(0.811.37)	1.07	(0.821.39)
STEMIc
Absent	Ref		Ref
Present	4.00	(2.755.81)	3.86	(2.645.63)
Troponin I
0.1 ng/ml	Ref		Ref
>0.1 ng/ml	1.01	(0.721.42)	1.02	(0.731.43)
COPS2d (AOR per 10 points)	1.05	(1.011.08)	1.04	(1.011.08)
LAPS2d (AOR per 10 points)	1.09	(1.061.11)	1.09	(1.061.11)
ED LOSe (hours)
<6	Ref		Ref
67	0.74	(0.531.03)	0.76	(0.541.06)
>=12	0.82	(0.391.74)	0.83	(0.391.78)
Code Statusf
Full Code	Ref
Not Full Code	1.08	(0.781.49)	1.09	(0.791.51)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.40	(0.290.54)	0.39	(0.280.53)
Number of therapies received
2 or less	Ref
3	0.49	(0.330.73)
4	0.29	(0.200.42)
5	0.17	(0.110.25)
Aspirin therapy			0.80	(0.491.32)
Anticoagulation therapy			0.86	(0.641.16)
Beta Blocker therapy			0.48	(0.360.64)
Statin therapy			0.63	(0.450.89)
ACE inhibitors or ARBs			0.40	(0.300.53)
C Statistic	0.814		0.822
Hosmer‐Lemeshow p value	0.509		0.934

Table 3.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Acute Myocardial Infarction Order Set
Outcome	Death
Number of outcomes	277
	AORa	95% CIb
a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized. g ^**See text for details on the order set.
Age in years
1839	Ref
4064	1.16	(0.158.78)
6584	4.67	(0.6334.46)
85+	5.45	(0.7340.86)
Sex
Female	Ref
Male	1.05	(0.811.36)
STEMIc
Absent	Ref
Present	3.86	(2.685.58)
Troponin I
0.1 ng/ml	Ref
>0.1 ng/ml	1.16	(0.831.62)
COPS2d (AOR per 10 points)	1.07	(1.031.10)
LAPS2d (AOR per 10 points)	1.09	(1.071.12)
ED LOSe (hours)
<6	Ref
67	0.72	(0.521.00)
>=12	0.70	(0.331.48)
Code statusf
Full code	Ref
Not full code	1.22	(0.891.68)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.37	(0.270.51)
Order set employedg
No	Ref
Yes	0.59	(0.450.76)
C Statistic	0.792
Hosmer‐Lemeshow p value	0.273

Table 4.Adjusted Odds Ratio (95% CI) or Mean Length‐of‐Stay Ratio (95% CI) in Study Patients
Outcome	Order Seta	3 Therapiesb	4 Therapiesb	5 Therapiesb
NOTE: Abbreviations: CI, confidence interval; LOS, length of stay. a Refers to comparison in which the reference group consists of patients who were not treated using the acute myocardial infarction order set. b Refers to comparison in which the reference group consists of patients who received 2 or less of the 5 recommended therapies. c See text for description of average treatment effect methodology. d See text for description of average treatment effect on the treated and matched pair adjustment methodology. e See text for details on how we modeled LOS.
Average treatment effectc
Inpatient mortality	0.67 (0.520.86)	0.64 (0.430.96)	0.37 (0.250.54)	0.23 (0.150.35)
30‐day mortality	0.77 (0.620.96)	0.68 (0.480.98)	0.34 (0.240.48)	0.26 (0.180.37)
Readmission	1.03 (0.901.19)	1.20 (0.871.66)	1.19 (0.881.60)	1.30 (0.961.76)
LOS, ratio of the geometric means	0.91 (0.870.95)	1.16 (1.031.30)	1.17 (1.051.30)	1.12 (1.001.24)
Average treatment effect on the treatedd
Inpatient mortality	0.69 (0.520.92)	0.35 (0.130.93)	0.17 (0.070.43)	0.08 (0.030.20)
30‐day mortality	0.84 (0.661.06)	0.35 (0.150.79)	0.17 (0.070.37)	0.09 (0.040.20)
Readmission	1.02 (0.871.20)	1.39 (0.852.26)	1.36 (0.882.12)	1.23 (0.801.89)
LOS, ratio of the geometric meanse	0.92 (0.870.97)	1.18 (1.021.37)	1.16 (1.011.33)	1.04 (0.911.19)

DISCUSSION

Acknowledgement

MATERIALS AND METHODS

This project was approved by the KPNC institutional review board.

Statistical Methods

We performed unadjusted comparisons between AMI‐OS and nonAMI‐OS patients using the t test or the [2] test, as appropriate.

RESULTS

Table 1.Description of Study Cohort
	Patients Initially Managed Using		P Valuea
	AMI Order Set, N=3,531b	A La Carte Orders, N=2,348b	P Valuea
NOTE: Abbreviations: ACE, angiotensin‐converting enzyme; AMI, acute myocardial infarction; AMI‐OS, acute myocardial infarction order set; ARBs, angiotensin receptor blockers; COPS2, Comorbidity Point Score, version 2; CPOE, computerized physician order entry; ED, emergency department; ICU, intensive care unit; LAPS2, Laboratory‐based Acute Physiology Score, version 2; SD, standard deviation; STEMI, ST‐segment elevation myocardial infarction. a ² or t test, as appropriate. See text for further methodological details. b AMI‐OS is an evidence‐based electronic checklist that guides physicians to order the most effective therapy by CPOE during the hospital admission process. In contrast, a la carte means that the clinician did not use the AMI‐OS, but rather entered individual orders via CPOE. See text for further details. c STEMI as evident by electrocardiogram. See text for details on ascertainment. d See text and reference 31 for details on how this score was assigned. e The COPS2 is a longitudinal, diagnosis‐based score assigned monthly that integrates all diagnoses incurred by a patient in the preceding 12 months. It is a continuous variable that can range between a minimum of zero and a theoretical maximum of 1,014, although <0.05% of Kaiser Permanente hospitalized patients have a COPS2 exceeding 241, and none have had a COPS2 >306. Increasing values of the COPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the COPS2. The LAPS2 integrates results from vital signs, neurological status checks, and 15 laboratory tests in the 72 hours preceding hospitalization into a single continuous variable. Increasing degrees of physiologic derangement are reflected in a higher LAPS2, which can range between a minimum of zero and a theoretical maximum of 414, although <0.05% of Kaiser Permanente hospitalized patients have a LAPS2 exceeding 227, and none have had a LAPS2 >282. Increasing values of LAPS2 are associated with increasing mortality. See text and references 20 and 27 for additional details on the LAPS2. f See text for details of specific therapies and how they were ascertained using the electronic medical record. g Percent mortality risk based on age, sex, diagnosis, COPS2, LAPS2, and care directive using a predictive model described in text and in reference 22. h See text for description of how end‐of‐life care directives are captured in the electronic medical record. i Direct admit means that the first hospital unit in which a patient stayed was the ICU; transfer refers to those patients transferred to the ICU from another unit in the hospital.
Age, y, median (meanSD)	70 (69.413.8)	70 (69.213.8)	0.5603
Age (% >65 years)	2,134 (60.4%)	1,415 (60.3%)	0.8949
Sex (% male)	2,202 (62.4%)	1,451 (61.8%)	0.6620
STEMI (% with)c	166 (4.7%)	369 (15.7%)	<0.0001
Troponin I (% missing)	111 (3.1%)	151 (6.4%)	<0.0001
Troponin I median (meanSD)	0.57 (3.08.2)	0.27 (2.58.9)	0.0651
Charlson score median (meanSD)d	2.0 (2.51.5)	2.0 (2.71.6)	<0.0001
COPS2, median (meanSD)e	14.0 (29.831.7)	17.0 (34.334.4)	<0.0001
LAPS2, median (meanSD)e	0.0 (35.643.5)	27.0 (40.948.1)	<0.0001
Length of stay in ED, h, median (meanSD)	5.7 (5.93.0)	5.7 (5.43.1)	<0.0001
Patients receiving aspirin within 24 hoursf	3,470 (98.3%)	2,202 (93.8%)	<0.0001
Patients receiving anticoagulation therapyf	2,886 (81.7%)	1,846 (78.6%)	0.0032
Patients receiving ‐blockersf	3,196 (90.5%)	1,926 (82.0%)	<0.0001
Patients receiving ACE inhibitors or ARBsf	2,395 (67.8%)	1,244 (53.0%)	<0.0001
Patients receiving statinsf	3,337 (94.5%)	1,975 (84.1%)	<0.0001
Patient received 1 or more therapies	3,531 (100.0%)	2,330 (99.2%)	<0.0001
Patient received 2 or more therapies	3,521 (99.7%)	2,266 (96.5%)	<0.0001
Patient received 3 or more therapies	3,440 (97.4%)	2,085 (88.8%)	<0.0001
Patient received 4 or more therapies	3,015 (85.4%)	1,646 (70.1%)	<0.0001
Patient received all 5 therapies	1,777 (50.3%)	866 (35.9%)	<0.0001
Predicted mortality risk, %, median, (meanSD)f	0.86 (3.27.4)	1.19 (4.810.8)	<0.0001
Full code at time of hospital entry (%)g	3,041 (86.1%)	2,066 (88.0%)	0.0379
Admitted to ICU (%)i
Direct admit	826 (23.4%)	567 (24.2%)	0.5047
Unplanned transfer	222 (6.3%)	133 (5.7%)	0.3262
Ever	1,283 (36.3%)	1,169 (49.8%)	<0.0001
Length of stay, h, median (meanSD)	68.3 (109.4140.9)	68.9 (113.8154.3)	0.2615
Inpatient mortality (%)	124 (3.5%)	153 (6.5%)	<0.0001
30‐day mortality (%)	200 (5.7%)	199 (8.5%)	<0.0001
All‐cause rehospitalization within 30 days (%)	576 (16.3%)	401 (17.1%)	0.4398
Cardiac catheterization procedure referral timing
1 day preadmission to discharge	2,018 (57.2%)	1,348 (57.4%)	0.1638
2 days preadmission or earlier	97 (2.8%)	87 (3.7%)
After discharge	149 (4.2%)	104 (4.4%)
No referral	1,267 (35.9%)	809 (34.5%)

Table 2.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Evidence‐Based Therapies
	Multiple Therapies Effect		Individual Therapies Effect
Outcome	Death		Death
Number of outcomes	277		277
	AORa	95% CIb	AORa	95% CIb
NOTE: Abbreviations: ACE = angiotensin converting enzyme; ARB = angiotensin receptor blockers. a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized.
Age in years
1839	Ref		Ref
4064	1.02	(0.147.73)	1.01	(0.137.66)
6584	4.05	(0.5529.72)	3.89	(0.5328.66)
85+	4.99	(0.6737.13)	4.80	(0.6435.84)
Sex
Female	Ref
Male	1.05	(0.811.37)	1.07	(0.821.39)
STEMIc
Absent	Ref		Ref
Present	4.00	(2.755.81)	3.86	(2.645.63)
Troponin I
0.1 ng/ml	Ref		Ref
>0.1 ng/ml	1.01	(0.721.42)	1.02	(0.731.43)
COPS2d (AOR per 10 points)	1.05	(1.011.08)	1.04	(1.011.08)
LAPS2d (AOR per 10 points)	1.09	(1.061.11)	1.09	(1.061.11)
ED LOSe (hours)
<6	Ref		Ref
67	0.74	(0.531.03)	0.76	(0.541.06)
>=12	0.82	(0.391.74)	0.83	(0.391.78)
Code Statusf
Full Code	Ref
Not Full Code	1.08	(0.781.49)	1.09	(0.791.51)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.40	(0.290.54)	0.39	(0.280.53)
Number of therapies received
2 or less	Ref
3	0.49	(0.330.73)
4	0.29	(0.200.42)
5	0.17	(0.110.25)
Aspirin therapy			0.80	(0.491.32)
Anticoagulation therapy			0.86	(0.641.16)
Beta Blocker therapy			0.48	(0.360.64)
Statin therapy			0.63	(0.450.89)
ACE inhibitors or ARBs			0.40	(0.300.53)
C Statistic	0.814		0.822
Hosmer‐Lemeshow p value	0.509		0.934

Table 3.Logistic Regression Model for Inpatient Mortality to Estimate the Effect of Acute Myocardial Infarction Order Set
Outcome	Death
Number of outcomes	277
	AORa	95% CIb
a Adjusted odds ratio. b 95% confidence interval. c ST‐segment elevation myocardial infarction present. d See text and preceding table for details on COmorbidity Point Score, version 2 and Laboratory Acute Physiology Score, version 2. e Emergency department length of stay. f See text for details on how care directives were categorized. g ^**See text for details on the order set.
Age in years
1839	Ref
4064	1.16	(0.158.78)
6584	4.67	(0.6334.46)
85+	5.45	(0.7340.86)
Sex
Female	Ref
Male	1.05	(0.811.36)
STEMIc
Absent	Ref
Present	3.86	(2.685.58)
Troponin I
0.1 ng/ml	Ref
>0.1 ng/ml	1.16	(0.831.62)
COPS2d (AOR per 10 points)	1.07	(1.031.10)
LAPS2d (AOR per 10 points)	1.09	(1.071.12)
ED LOSe (hours)
<6	Ref
67	0.72	(0.521.00)
>=12	0.70	(0.331.48)
Code statusf
Full code	Ref
Not full code	1.22	(0.891.68)
Cardiac procedure referral
None during stay	Ref
1 day pre adm until discharge	0.37	(0.270.51)
Order set employedg
No	Ref
Yes	0.59	(0.450.76)
C Statistic	0.792
Hosmer‐Lemeshow p value	0.273

Table 4.Adjusted Odds Ratio (95% CI) or Mean Length‐of‐Stay Ratio (95% CI) in Study Patients
Outcome	Order Seta	3 Therapiesb	4 Therapiesb	5 Therapiesb
NOTE: Abbreviations: CI, confidence interval; LOS, length of stay. a Refers to comparison in which the reference group consists of patients who were not treated using the acute myocardial infarction order set. b Refers to comparison in which the reference group consists of patients who received 2 or less of the 5 recommended therapies. c See text for description of average treatment effect methodology. d See text for description of average treatment effect on the treated and matched pair adjustment methodology. e See text for details on how we modeled LOS.
Average treatment effectc
Inpatient mortality	0.67 (0.520.86)	0.64 (0.430.96)	0.37 (0.250.54)	0.23 (0.150.35)
30‐day mortality	0.77 (0.620.96)	0.68 (0.480.98)	0.34 (0.240.48)	0.26 (0.180.37)
Readmission	1.03 (0.901.19)	1.20 (0.871.66)	1.19 (0.881.60)	1.30 (0.961.76)
LOS, ratio of the geometric means	0.91 (0.870.95)	1.16 (1.031.30)	1.17 (1.051.30)	1.12 (1.001.24)
Average treatment effect on the treatedd
Inpatient mortality	0.69 (0.520.92)	0.35 (0.130.93)	0.17 (0.070.43)	0.08 (0.030.20)
30‐day mortality	0.84 (0.661.06)	0.35 (0.150.79)	0.17 (0.070.37)	0.09 (0.040.20)
Readmission	1.02 (0.871.20)	1.39 (0.852.26)	1.36 (0.882.12)	1.23 (0.801.89)
LOS, ratio of the geometric meanse	0.92 (0.870.97)	1.18 (1.021.37)	1.16 (1.011.33)	1.04 (0.911.19)

DISCUSSION

Acknowledgement

References

Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Rosamond WD, Chambless LE, Heiss G, et al. Twenty‐two‐year trends in incidence of myocardial infarction, coronary heart disease mortality, and case fatality in 4 US communities, 1987–2008. Circulation. 2012;125(15):1848–1857.
Roger VL, Go AS, Lloyd‐Jones DM, et al. Heart disease and stroke statistics—2012 update: a report from the American Heart Association. Circulation. 2012;125(1):e2–e220.
Anderson JL, Adams CD, Antman EM, et al. ACC/AHA 2007 guidelines for the management of patients with unstable angina/non‐ST‐Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non‐ST‐Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007;50(7):e1–e157.
Antman EM, Hand M, Armstrong PW, et al. 2007 focused update of the ACC/AHA 2004 guidelines for the management of patients with ST‐elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2008;51(2):210–247.
Jernberg T, Johanson P, Held C, Svennblad B, Lindback J, Wallentin L. Association between adoption of evidence‐based treatment and survival for patients with ST‐elevation myocardial infarction. JAMA. 2011;305(16):1677–1684.
Puymirat E, Simon T, Steg PG, et al. Association of changes in clinical characteristics and management with improvement in survival among patients with ST‐elevation myocardial infarction. JAMA. 2012;308(10):998–1006.
Motivala AA, Cannon CP, Srinivas VS, et al. Changes in myocardial infarction guideline adherence as a function of patient risk: an end to paradoxical care? J Am Coll Cardiol. 2011;58(17):1760–1765.
Jha AK, Li Z, Orav EJ, Epstein AM. Care in U.S. hospitals—the Hospital Quality Alliance program. N Engl J Med. 2005;353(3):265–274.
Desai N, Chen AN, et al. Challenges in the treatment of NSTEMI patients at high risk for both ischemic and bleeding events: insights from the ACTION Registry‐GWTG. J Am Coll Cardiol. 2011;57:E913–E913.
Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
Eagle KA, Montoye K, Riba AL. Guideline‐based standardized care is associated with substantially lower mortality in medicare patients with acute myocardial infarction. J Am Coll Cardiol. 2005;46(7):1242–1248.
Ballard DJ, Ogola G, Fleming NS, et al. Impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care. Int J Qual Health Care. 2010;22(6):437–444.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48(8):739–744.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
International Classification of Diseases, 9th Revision‐Clinical Modification. 4th ed. 3 Vols. Los Angeles, CA: Practice Management Information Corporation; 2006.
Go AS, Hylek EM, Chang Y, et al. Anticoagulation therapy for stroke prevention in atrial fibrillation: how well do randomized trials translate into clinical practice? JAMA. 2003;290(20):2685–2692.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, David D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Kipnis P, Escobar GJ, Draper D. Effect of choice of estimation method on inter‐hospital mortality rate comparisons. Med Care. 2010;48(5):456–485.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63(7):798–803.
Wong J, Taljaard M, Forster AJ, Escobar GJ, Walraven C. Derivation and validation of a model to predict daily risk of death in hospital. Med Care. 2011;49(8):734–743.
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
MacKinnon DP. Introduction to Statistical Mediation Analysis. New York, NY: Lawrence Erlbaum Associates; 2008.
Imbens GW. Nonparametric estimation of average treatment effects under exogenity: a review. Rev Econ Stat. 2004;86:25.
Rosenbaum PR. Design of Observational Studies. New York, NY: Springer Science+Business Media; 2010.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Stat Med. 2009;28:24.
Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994(89):846–866.
Lunceford JK, Davidian M. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med. 2004;23(19):2937–2960.
Rosenbaum PR. Discussing hidden bias in observational studies. Ann Intern Med. 1991;115(11):901–905.
D'Agostino RB. Propensity score methods for bias reduction in the comparison of a treatment to a non‐randomized control group. Stat Med. 1998;17(19):2265–2281.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study, 2005. www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed on September 14, 2013.
Ettinger WH. Using health information technology to improve health care. Arch Intern Med. 2012;172(22):1728–1730.

References

Yeh RW, Sidney S, Chandra M, Sorel M, Selby JV, Go AS. Population trends in the incidence and outcomes of acute myocardial infarction. N Engl J Med. 2010;362(23):2155–2165.
Rosamond WD, Chambless LE, Heiss G, et al. Twenty‐two‐year trends in incidence of myocardial infarction, coronary heart disease mortality, and case fatality in 4 US communities, 1987–2008. Circulation. 2012;125(15):1848–1857.
Roger VL, Go AS, Lloyd‐Jones DM, et al. Heart disease and stroke statistics—2012 update: a report from the American Heart Association. Circulation. 2012;125(1):e2–e220.
Anderson JL, Adams CD, Antman EM, et al. ACC/AHA 2007 guidelines for the management of patients with unstable angina/non‐ST‐Elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Revise the 2002 Guidelines for the Management of Patients With Unstable Angina/Non‐ST‐Elevation Myocardial Infarction) developed in collaboration with the American College of Emergency Physicians, the Society for Cardiovascular Angiography and Interventions, and the Society of Thoracic Surgeons endorsed by the American Association of Cardiovascular and Pulmonary Rehabilitation and the Society for Academic Emergency Medicine. J Am Coll Cardiol. 2007;50(7):e1–e157.
Antman EM, Hand M, Armstrong PW, et al. 2007 focused update of the ACC/AHA 2004 guidelines for the management of patients with ST‐elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2008;51(2):210–247.
Jernberg T, Johanson P, Held C, Svennblad B, Lindback J, Wallentin L. Association between adoption of evidence‐based treatment and survival for patients with ST‐elevation myocardial infarction. JAMA. 2011;305(16):1677–1684.
Puymirat E, Simon T, Steg PG, et al. Association of changes in clinical characteristics and management with improvement in survival among patients with ST‐elevation myocardial infarction. JAMA. 2012;308(10):998–1006.
Motivala AA, Cannon CP, Srinivas VS, et al. Changes in myocardial infarction guideline adherence as a function of patient risk: an end to paradoxical care? J Am Coll Cardiol. 2011;58(17):1760–1765.
Jha AK, Li Z, Orav EJ, Epstein AM. Care in U.S. hospitals—the Hospital Quality Alliance program. N Engl J Med. 2005;353(3):265–274.
Desai N, Chen AN, et al. Challenges in the treatment of NSTEMI patients at high risk for both ischemic and bleeding events: insights from the ACTION Registry‐GWTG. J Am Coll Cardiol. 2011;57:E913–E913.
Williams SC, Schmaltz SP, Morton DJ, Koss RG, Loeb JM. Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
Eagle KA, Montoye K, Riba AL. Guideline‐based standardized care is associated with substantially lower mortality in medicare patients with acute myocardial infarction. J Am Coll Cardiol. 2005;46(7):1242–1248.
Ballard DJ, Ogola G, Fleming NS, et al. Impact of a standardized heart failure order set on mortality, readmission, and quality and costs of care. Int J Qual Health Care. 2010;22(6):437–444.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar G, Greene J, Scheirer P, Gardner M, Draper D, Kipnis P. Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48(8):739–744.
Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6(2):74–80.
Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224–230.
International Classification of Diseases, 9th Revision‐Clinical Modification. 4th ed. 3 Vols. Los Angeles, CA: Practice Management Information Corporation; 2006.
Go AS, Hylek EM, Chang Y, et al. Anticoagulation therapy for stroke prevention in atrial fibrillation: how well do randomized trials translate into clinical practice? JAMA. 2003;290(20):2685–2692.
Escobar GJ, LaGuardia J, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med. 2012;7(5):388–395.
Escobar GJ, Gardner M, Greene JG, David D, Kipnis P. Risk‐adjusting hospital mortality using a comprehensive electronic record in an integrated healthcare delivery system. Med Care. 2013;51(5):446–453.
Kipnis P, Escobar GJ, Draper D. Effect of choice of estimation method on inter‐hospital mortality rate comparisons. Med Care. 2010;48(5):456–485.
Walraven C, Escobar GJ, Greene JD, Forster AJ. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2010;63(7):798–803.
Wong J, Taljaard M, Forster AJ, Escobar GJ, Walraven C. Derivation and validation of a model to predict daily risk of death in hospital. Med Care. 2011;49(8):734–743.
Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
MacKinnon DP. Introduction to Statistical Mediation Analysis. New York, NY: Lawrence Erlbaum Associates; 2008.
Imbens GW. Nonparametric estimation of average treatment effects under exogenity: a review. Rev Econ Stat. 2004;86:25.
Rosenbaum PR. Design of Observational Studies. New York, NY: Springer Science+Business Media; 2010.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity‐score matched samples. Stat Med. 2009;28:24.
Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994(89):846–866.
Lunceford JK, Davidian M. Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat Med. 2004;23(19):2937–2960.
Rosenbaum PR. Discussing hidden bias in observational studies. Ann Intern Med. 1991;115(11):901–905.
D'Agostino RB. Propensity score methods for bias reduction in the comparison of a treatment to a non‐randomized control group. Stat Med. 1998;17(19):2265–2281.
Feng WW, Jun Y, Xu R. A method/macro based on propensity score and Mahalanobis distance to reduce bias in treatment comparison in observational study, 2005. www.lexjansen.com/pharmasug/2006/publichealthresearch/pr05.pdf. Accessed on September 14, 2013.
Ettinger WH. Using health information technology to improve health care. Arch Intern Med. 2012;172(22):1728–1730.

Issue

Journal of Hospital Medicine - 9(3)

Issue

Journal of Hospital Medicine - 9(3)

Page Number

155-161

Page Number

155-161

Publications

Publications

Article Type

Display Headline

An electronic order set for acute myocardial infarction is associated with improved patient outcomes through better adherence to clinical practice guidelines

Display Headline

An electronic order set for acute myocardial infarction is associated with improved patient outcomes through better adherence to clinical practice guidelines

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Address for correspondence and reprint requests: Gabriel J. Escobar, MD, Division of Research, Kaiser Permanente Northern California, 2000 Broadway Avenue, 032R01, Oakland, CA 94612; Telephone: 510‐891‐5929; E‐mail: gabriel.escobar@kp.org

Content Gating

No Gating (article Unlocked/Free)

Alternative CME

Article PDF Media

Media Files

Risk Factors For Unplanned ICU Transfer

Article Type

Changed

Mon, 05/22/2017 - 18:05

Display Headline

Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system

Author(s)

Jesse M. Pines, MD, MBA, MSCE

Marla N. Gardner, BA

Emergency Department (ED) patients who are hospitalized and require unplanned transfer to the intensive care unit (ICU) within 24 hours of arrival on the ward have higher mortality than direct ICU admissions.1, 2 Previous research found that 5% of ED admissions experienced unplanned ICU transfer during their hospitalization, yet these patients account for 25% of in‐hospital deaths and have a longer length of stay than direct ICU admissions.1, 3 For these reasons, inpatient rapid‐response teams and early warning systems have been studied to reduce the mortality of patients who rapidly deteriorate on the hospital ward.410 However, there is little conclusive evidence that these interventions decrease mortality.710 It is possible that with better recognition and intervention in the ED, a portion of these unplanned ICU transfers and their subsequent adverse outcomes could be prevented.11

Previous research on risk factors for unplanned ICU transfers among ED admissions is limited. While 2 previous studies from non‐US hospitals used administrative data to identify some general populations at risk for unplanned ICU transfer,12, 13 these studies did not differentiate between transfers shortly after admission and those that occurred during a prolonged hospital staya critical distinction since the outcomes between these groups differs substantially.1 Another limitation of these studies is the absence of physiologic measures at ED presentation, which have been shown to be highly predictive of mortality.14

In this study, we describe risk factors for unplanned transfer to the ICU within 24 hours of arrival on the ward, among a large cohort of ED hospitalizations across 13 community hospitals. Focusing on admitting diagnoses most at risk, our goal was to inform efforts to improve the triage of ED admissions and determine which patients may benefit from additional interventions, such as improved resuscitation, closer monitoring, or risk stratification tools. We also hypothesized that higher volume hospitals would have lower rates of unplanned ICU transfers, as these hospitals are more likely have more patient care resources on the hospital ward and a higher threshold to transfer to the ICU.

METHODS

Setting and Patients

The setting for this study was Kaiser Permanente Northern California (KPNC), a large integrated healthcare delivery system serving approximately 3.3 million members.1, 3, 15, 16 We extracted data on all adult ED admissions (18 years old) to the hospital between 2007 and 2009. We excluded patients who went directly to the operating room or the ICU, as well as gynecological/pregnancy‐related admissions, as these patients have substantially different mortality risks.14 ED admissions to hospital wards could either go to medicalsurgical units or transitional care units (TCU), an intermediate level of care between the medicalsurgical units and the ICU. We chose to focus on hospitals with similar inpatient structures. Thus, 8 hospitals without TCUs were excluded, leaving 13 hospitals for analysis. The KPNC Institutional Review Board approved this study.

Main Outcome Measure

The main outcome measure was unplanned transfer to the ICU within 24 hours of arrival to the hospital ward, based upon bed history data. As in previous research, we make the assumptionwhich is supported by the high observed‐to‐expected mortality ratios found in these patientsthat these transfers to the ICU were due to clinical deterioration, and thus were unplanned, rather than a planned transfer to the ICU as is more common after an elective surgical procedure.13 The comparison population was patients admitted from the ED to the ward who never experienced a transfer to the ICU.

Patient and Hospital Characteristics

We extracted patient data on age, sex, admitting diagnosis, chronic illness burden, acute physiologic derangement in the ED, and hospital unit length of stay. Chronic illness was measured using the Comorbidity Point Score (COPS), and physiologic derangement was measured using the Laboratory Acute Physiology Score (LAPS) calculated from labs collected in the ED.1, 14, 17 The derivation of these variables from the electronic medical record has been previously described.14 The COPS was derived from International Classification of Diseases, Ninth Revision (ICD‐9) codes for all Kaiser Permanente Medical Care Program (KPMCP) inpatient and outpatient encounters prior to hospitalization. The LAPS is based on 14 possible lab tests that could be drawn in the ED or in the 72 hours prior to hospitalization. The admitting diagnosis is the ICD‐9 code assigned for the primary diagnosis determined by the admitting physician at the time when hospital admission orders are entered. We further collapsed a previously used categorization of 44 primary condition diagnoses, based on admission ICD‐9 codes,14 into 25 broad diagnostic categories based on pathophysiologic plausibility and mortality rates. We tabulated inpatient admissions originating in the ED to derive a hospital volume measure.

Statistical Analyses

We compared patient characteristics, hospital volume, and outcomes by whether or not an unplanned ICU transfer occurred. Unadjusted analyses were performed with analysis of variance (ANOVA) and chi‐square tests. We calculated crude rates of unplanned ICU transfer per 1,000 ED inpatient admissions by patient characteristics and by hospital, stratified by hospital volume.

We used a hierarchical multivariate logistic regression model to estimate adjusted odds ratios for unplanned ICU transfer as a function of both patient‐level variables (age, sex, COPS, LAPS, time of admission, admission to TCU vs ward, admitting diagnosis) and hospital‐level variables (volume) in the model. We planned to choose the reference group for admitting diagnosis as the one with an unadjusted odds ratio closest to the null (1.00). This model addresses correlations between patients with multiple hospitalizations and clustering by hospital, by fitting random intercepts for these clusters. All analyses were performed in Stata 12 (StataCorp, College Station, TX), and statistics are presented with 95% confidence intervals (CI). The Stata program gllamm (Generalized Linear Latent and Mixed Models) was used for hierarchical modeling.18

RESULTS

Of 178,315 ED non‐ICU hospitalizations meeting inclusion criteria, 4,252 (2.4%) were admitted to the ward and were transferred to the ICU within 24 hours of leaving the ED. There were 122,251 unique patients in our study population. Table 1 compares the characteristics of ED hospitalizations in which an unplanned transfer occurred to those that did not experience an unplanned transfer. Unplanned transfers were more likely to have a higher comorbidity burden, more deranged physiology, and more likely to arrive on the floor during the overnight shift.

Table 1.Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
Characteristics	Unplanned Transfer to ICU Within 24 h of Leaving ED?		P Value*
Characteristics	Yes	No	P Value*
N = 4,252 (2.4%)	N = 174,063 (97.6%)
Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation. * P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details. Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.
Age, median (IQR)	69 (5680)	70 (5681)	<0.01
Male, %	51.3	45.9	<0.01
Comorbidity Points Score (COPS), median (IQR)	100 (46158)	89 (42144)	<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)	26 (1342)	18 (633)	<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)	20.1	20.1	NS
Evening: 3 pm11 pm	47.6	50.2	NS
Overnight: 11 pm7 am	32.3	29.7	<0.01
Weekend admission, %	33.7	32.7	NS
Admitted to monitored bed, %	24.1	24.9	NS
Emergency department annual volume, mean (SD)	48,755 (15,379)	50,570 (15,276)	<0.01
Non‐ICU annual admission volume, mean (SD)	5,562 (1,626)	5,774 (1,568)	<0.01
Admitting diagnosis, listed by descending frequency, %			NS
Pneumonia and respiratory infections	16.3	11.8	<0.01
Gastrointestinal bleeding	12.8	13.6	NS
Chest pain	7.3	10.0	<0.01
Miscellaneous conditions	5.6	6.2	NS
All other acute infections	4.7	6.0	<0.01
Seizures	4.1	5.9	<0.01
AMI	3.9	3.3	<0.05
COPD	3.8	3.0	<0.01
CHF	3.5	3.7	NS
Arrhythmias and pulmonary embolism	3.5	3.3	NS
Stroke	3.4	3.5	NS
Diabetic emergencies	3.3	2.6	<0.01
Metabolic, endocrine, electrolytes	3.0	2.9	NS
Sepsis	3.0	1.2	<0.01
Other neurology and toxicology	3.0	2.9	NS
Urinary tract infections	2.9	3.2	NS
Catastrophic conditions	2.6	1.2	<0.01
Rheumatology	2.5	3.5	<0.01
Hematology and oncology	2.4	2.4	NS
Acute renal failure	1.9	1.1	<0.01
Pancreatic and liver	1.7	2.0	NS
Trauma, fractures, and dislocations	1.6	1.8	NS
Bowel obstructions and diseases	1.6	2.9	<0.01
Other cardiac conditions	1.5	1.3	NS
Other renal conditions	0.6	1.0	<0.01
Inpatient length of stay, median days (IQR)	4.7 (2.78.6)	2.6 (1.54.4)	<0.01
Died during hospitalization, %	12.7	2.4	<0.01

Unplanned ICU transfers were more frequent in lower volume hospitals (Table 1). Figure 1 displays the inverse relationship between hospital annual ED inpatient admission volume and unplanned ICU transfers rates. The lowest volume hospital had a crude rate twice as high as the 2 highest volume hospitals (39 vs 20, per 1,000 admissions).

Relationship between hospital volume and rate of unplanned ICU transfers within 24 hours. Abbreviations: ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

Pneumonia/respiratory infection was the most frequent admitting condition associated with unplanned transfer (16.3%) (Table 1). There was also wide variation in crude rates for unplanned ICU transfer by admitting condition (Figure 2). Patients admitted with sepsis had the highest rate (59 per 1,000 admissions), while patients admitted with renal conditions other than acute renal failure had the lowest rates (14.3 per 1,000 admissions).

Association between patient characteristics, hospital volume, and risk of unplanned ICU transfer within 24 hours in a hierarchical logistic regression model. Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

We confirmed that almost all diagnoses found to account for a disproportionately high share of unplanned ICU transfers in Table 1 were indeed independently associated with this phenomenon after adjustment for patient and hospital differences (Figure 2). Pneumonia remained the most frequent condition associated with unplanned ICU transfer (odds ratio [OR] 1.50; 95% CI 1.201.86). Although less frequent, sepsis had the strongest association of any condition with unplanned transfer (OR 2.51; 95% CI 1.903.31). However, metabolic, endocrine, and electrolyte conditions were no longer associated with unplanned transfer after adjustment, while arrhythmias and pulmonary embolism were. Other conditions confirmed to be associated with increased risk of unplanned transfer included: myocardial infarction (MI), chronic obstructive pulmonary disease (COPD), stroke, diabetic emergencies, catastrophic conditions (includes aortic catastrophes, all forms of shock except septic shock, and intracranial hemorrhage), and acute renal failure. After taking into account the frequency of admitting diagnoses, respiratory conditions (COPD, pneumonia/acute respiratory infection) comprised nearly half (47%) of all conditions associated with increased risk of unplanned ICU transfer.

Other factors confirmed to be independently associated with unplanned ICU transfer included: male sex (OR 1.20; 95% CI 1.131.28), high comorbidity burden as measured by COPS >145 (OR 1.13; 95% CI 1.031.24), increasingly abnormal physiology compared to a LAPS <7, and arrival on ward during the overnight shift (OR 1.10; 95% CI 1.011.21). After adjustment, we did find that admission to the TCU rather than a medicalsurgical unit was associated with decreased risk of unplanned ICU transfer (OR 0.83; 95% CI 0.770.90). Age 85 was associated with decreased risk of unplanned ICU transfer relative to the youngest age group of 1834‐year‐old patients (OR 0.64; 95% CI 0.530.77).

ED admissions to higher volume hospitals were 6% less likely to experience an unplanned transfer for each additional 1,000 annual ED hospitalizations over a lower volume hospital (OR 0.94; 95% CI 0.910.98). In other words, a patient admitted to a hospital with 8,000 annual ED hospitalizations had 30% decreased odds of unplanned ICU transfer compared to a hospital with only 3,000 annual ED hospitalizations.

DISCUSSION

Patients admitted with respiratory conditions accounted for half of all admitting diagnoses associated with increased risk of unplanned transfer to the ICU within 24 hours of arrival to the ward. We found that 1 in 30 ED ward admissions for pneumonia, and 1 in 33 for COPD, were transferred to the ICU within 24 hours. These findings indicate that there is some room for improvement in early care of respiratory conditions, given the average unplanned transfer rate of 1 in 42, and previous research showing that patients with pneumonia and patients with COPD, who experience unplanned ICU transfer, have substantially worse mortality than those directly admitted to the ICU.1

Although less frequent than hospitalizations for respiratory conditions, patients admitted with sepsis were at the highest risk of unplanned ICU transfer (1 in 17 ED non‐ICU hospitalizations). We also found that MI and stroke ward admissions had a higher risk of unplanned ICU transfer. However, we previously found that unplanned ICU transfers for sepsis, MI, and stroke did not have worse mortality than direct ICU admits for these conditions.1 Therefore, quality improvement efforts to reduce excess mortality related to early decompensation in the hospital and unplanned ICU transfer would be most effective if targeted towards respiratory conditions such as pneumonia and COPD.

This is the only in‐depth study, to our knowledge, to explore the association between a set of mutually exclusive diagnostic categories and risk of unplanned ICU transfer within 24 hours, and it is the first study to identify risk factors for unplanned ICU transfer in a multi‐hospital cohort adjusted for patient‐ and hospital‐level characteristics. We also identified a novel hospital volumeoutcome relationship: Unplanned ICU transfers are up to twice as likely to occur in the smallest volume hospitals compared with highest volume hospitals. Hospital volume has long been proposed as a proxy for hospital resources; there are several studies showing a relationship between low‐volume hospitals and worse outcomes for a number of conditions.19, 20 Possible mechanisms may include decreased ICU capacity, decreased on‐call intensivists in the hospital after hours, and less experience with certain critical care conditions seen more frequently in high‐volume hospitals.21

Patients at risk of unplanned ICU transfer were also more likely to have physiologic derangement identified on laboratory testing, high comorbidity burden, and arrive on the ward between 11 PM and 7 AM. Given the strong correlation between comorbidity burden and physiologic derangement and mortality,14 it is not surprising that the COPS and LAPS were independent predictors of unplanned transfer. It is unclear, however, why arriving on the ward on the overnight shift is associated with higher risk. One possibility is that patients who arrive on the wards during 11 PM to 7 AM are also likely to have been in the ED during evening peak hours most associated with ED crowding.22 High levels of ED crowding have been associated with delays in care, worse quality care, lapses in patient safety, and even increased in‐hospital mortality.22, 23 Other possible reasons include decreased in‐hospital staffing and longer delays in critical diagnostic tests and interventions.2428

Admission to TCUs was associated with decreased risk of unplanned ICU transfer in the first 24 hours of hospitalization. This may be due to the continuous monitoring, decreased nursing‐to‐patient ratios, or the availability to provide some critical care interventions. In our study, age 85 was associated with lower likelihood of unplanned transfer. Unfortunately, we did not have access to data on advanced directives or patient preferences. Data on advanced directives would help to distinguish whether this phenomenon was related to end‐of‐life care goals versus other explanations.

Our study confirms some risk factors identified in previous studies. These include specific diagnoses such as pneumonia and COPD,12, 13, 29 heavy comorbidity burden,12, 13, 29 abnormal labs,29 and male sex.13 Pneumonia has consistently been shown to be a risk factor for unplanned ICU transfer. This may stem from the dynamic nature of this condition and its ability to rapidly progress, and the fact that some ICUs may not accept pneumonia patients unless they demonstrate a need for mechanical ventilation.30 Recently, a prediction rule has been developed to determine which patients with pneumonia are likely to have an unplanned ICU transfer.30 It is possible that with validation and application of this rule, unplanned transfer rates for pneumonia could be reduced. It is unclear whether males have unmeasured factors associated with increased risk of unplanned transfer or whether a true gender disparity exists.

Our findings should be interpreted within the context of this study's limitations. First, this study was not designed to distinguish the underlying cause of the unplanned transfer such as under‐recognition of illness severity in the ED, evolving clinical disease after leaving the ED, or delays in critical interventions on the ward. These are a focus of our ongoing research efforts. Second, while previous studies have demonstrated that our automated risk adjustment variables can accurately predict in‐hospital mortality (0.88 area under curve in external populations),17 additional data on vital signs and mental status could further improve risk adjustment. However, using automated data allowed us to study risk factors for unplanned transfer in a multi‐hospital cohort with a much larger population than has been previously studied. Serial data on vital signs and mental status both in the ED and during hospitalization could also be helpful in determining which unplanned transfers could be prevented with earlier recognition and intervention. Finally, all patient care occurred within an integrated healthcare delivery system. Thus, differences in case‐mix, hospital resources, ICU structure, and geographic location should be considered when applying our results to other healthcare systems.

This study raises several new areas for future research. With access to richer data becoming available in electronic medical records, prediction rules should be developed to enable better triage to appropriate levels of care for ED admissions. Future research should also analyze the comparative effectiveness of intermediate monitored units versus non‐monitored wards for preventing clinical deterioration by admitting diagnosis. Diagnoses that have been shown to have an increased risk of death after unplanned ICU transfer, such as pneumonia/respiratory infection and COPD,1 should be prioritized in this research. Better understanding is needed on the diagnosis‐specific differences and the differences in ED triage process and ICU structure that may explain why high‐volume hospitals have significantly lower rates of early unplanned ICU transfers compared with low‐volume hospitals. In particular, determining the effect of TCU and ICU capacities and census at the time of admission, and comparing patient risk characteristics across hospital‐volume strata would be very useful. Finally, more work is needed to determine whether the higher rate of unplanned transfers during overnight nursing shifts is related to decreased resource availability, preceding ED crowding, or other organizational causes.

In conclusion, patients admitted with respiratory conditions, sepsis, MI, high comorbidity, and abnormal labs are at modestly increased risk of unplanned ICU transfer within 24 hours of admission from the ED. Patients admitted with respiratory conditions (pneumonia/respiratory infections and COPD) accounted for half of the admitting diagnoses that are at increased risk for unplanned ICU transfer. These patients may benefit from better inpatient triage from the ED, earlier intervention, or closer monitoring. More research is needed to determine the specific aspects of care associated with admission to intermediate care units and high‐volume hospitals that reduce the risk of unplanned ICU transfer.

Acknowledgements

The authors thank John D. Greene, Juan Carlos La Guardia, and Benjamin Turk for their assistance with formatting of the dataset; Dr Alan S. Go, Acting Director of the Division of Research, for reviewing the manuscript; and Alina Schnake‐Mahl for formatting the manuscript.

References

Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224–230.
Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):77–83.
Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74–80.
Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):2267–2274.
Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):2091–2097.
Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):1238–1243.
Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422–432.
Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):18–26.
McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:68–72.
Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241–248.
Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224–230.
Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798–803.
Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301–323.
Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):1677–1679.
Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511–520.
Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126–136.
Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10.
Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):68–75.
Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569–576.
Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803–812.
Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317–324.
Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):1489–1495.
Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):1080–1085.
Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.

Article PDF

Issue

Journal of Hospital Medicine - 8(1)

Publications

Page Number

13-19

Read more about Risk Factors For Unplanned ICU Transfer

Sections

Author(s)

Jesse M. Pines, MD, MBA, MSCE

Marla N. Gardner, BA

Author(s)

Jesse M. Pines, MD, MBA, MSCE

Marla N. Gardner, BA

Article PDF

Article PDF

METHODS

Setting and Patients

Main Outcome Measure

Patient and Hospital Characteristics

Statistical Analyses

RESULTS

Table 1.Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
Characteristics	Unplanned Transfer to ICU Within 24 h of Leaving ED?		P Value*
Characteristics	Yes	No	P Value*
N = 4,252 (2.4%)	N = 174,063 (97.6%)
Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation. * P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details. Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.
Age, median (IQR)	69 (5680)	70 (5681)	<0.01
Male, %	51.3	45.9	<0.01
Comorbidity Points Score (COPS), median (IQR)	100 (46158)	89 (42144)	<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)	26 (1342)	18 (633)	<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)	20.1	20.1	NS
Evening: 3 pm11 pm	47.6	50.2	NS
Overnight: 11 pm7 am	32.3	29.7	<0.01
Weekend admission, %	33.7	32.7	NS
Admitted to monitored bed, %	24.1	24.9	NS
Emergency department annual volume, mean (SD)	48,755 (15,379)	50,570 (15,276)	<0.01
Non‐ICU annual admission volume, mean (SD)	5,562 (1,626)	5,774 (1,568)	<0.01
Admitting diagnosis, listed by descending frequency, %			NS
Pneumonia and respiratory infections	16.3	11.8	<0.01
Gastrointestinal bleeding	12.8	13.6	NS
Chest pain	7.3	10.0	<0.01
Miscellaneous conditions	5.6	6.2	NS
All other acute infections	4.7	6.0	<0.01
Seizures	4.1	5.9	<0.01
AMI	3.9	3.3	<0.05
COPD	3.8	3.0	<0.01
CHF	3.5	3.7	NS
Arrhythmias and pulmonary embolism	3.5	3.3	NS
Stroke	3.4	3.5	NS
Diabetic emergencies	3.3	2.6	<0.01
Metabolic, endocrine, electrolytes	3.0	2.9	NS
Sepsis	3.0	1.2	<0.01
Other neurology and toxicology	3.0	2.9	NS
Urinary tract infections	2.9	3.2	NS
Catastrophic conditions	2.6	1.2	<0.01
Rheumatology	2.5	3.5	<0.01
Hematology and oncology	2.4	2.4	NS
Acute renal failure	1.9	1.1	<0.01
Pancreatic and liver	1.7	2.0	NS
Trauma, fractures, and dislocations	1.6	1.8	NS
Bowel obstructions and diseases	1.6	2.9	<0.01
Other cardiac conditions	1.5	1.3	NS
Other renal conditions	0.6	1.0	<0.01
Inpatient length of stay, median days (IQR)	4.7 (2.78.6)	2.6 (1.54.4)	<0.01
Died during hospitalization, %	12.7	2.4	<0.01

DISCUSSION

Acknowledgements

METHODS

Setting and Patients

Main Outcome Measure

Patient and Hospital Characteristics

Statistical Analyses

RESULTS

Table 1.Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
Characteristics	Unplanned Transfer to ICU Within 24 h of Leaving ED?		P Value*
Characteristics	Yes	No	P Value*
N = 4,252 (2.4%)	N = 174,063 (97.6%)
Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation. * P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details. Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.
Age, median (IQR)	69 (5680)	70 (5681)	<0.01
Male, %	51.3	45.9	<0.01
Comorbidity Points Score (COPS), median (IQR)	100 (46158)	89 (42144)	<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)	26 (1342)	18 (633)	<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)	20.1	20.1	NS
Evening: 3 pm11 pm	47.6	50.2	NS
Overnight: 11 pm7 am	32.3	29.7	<0.01
Weekend admission, %	33.7	32.7	NS
Admitted to monitored bed, %	24.1	24.9	NS
Emergency department annual volume, mean (SD)	48,755 (15,379)	50,570 (15,276)	<0.01
Non‐ICU annual admission volume, mean (SD)	5,562 (1,626)	5,774 (1,568)	<0.01
Admitting diagnosis, listed by descending frequency, %			NS
Pneumonia and respiratory infections	16.3	11.8	<0.01
Gastrointestinal bleeding	12.8	13.6	NS
Chest pain	7.3	10.0	<0.01
Miscellaneous conditions	5.6	6.2	NS
All other acute infections	4.7	6.0	<0.01
Seizures	4.1	5.9	<0.01
AMI	3.9	3.3	<0.05
COPD	3.8	3.0	<0.01
CHF	3.5	3.7	NS
Arrhythmias and pulmonary embolism	3.5	3.3	NS
Stroke	3.4	3.5	NS
Diabetic emergencies	3.3	2.6	<0.01
Metabolic, endocrine, electrolytes	3.0	2.9	NS
Sepsis	3.0	1.2	<0.01
Other neurology and toxicology	3.0	2.9	NS
Urinary tract infections	2.9	3.2	NS
Catastrophic conditions	2.6	1.2	<0.01
Rheumatology	2.5	3.5	<0.01
Hematology and oncology	2.4	2.4	NS
Acute renal failure	1.9	1.1	<0.01
Pancreatic and liver	1.7	2.0	NS
Trauma, fractures, and dislocations	1.6	1.8	NS
Bowel obstructions and diseases	1.6	2.9	<0.01
Other cardiac conditions	1.5	1.3	NS
Other renal conditions	0.6	1.0	<0.01
Inpatient length of stay, median days (IQR)	4.7 (2.78.6)	2.6 (1.54.4)	<0.01
Died during hospitalization, %	12.7	2.4	<0.01

DISCUSSION

Acknowledgements

References

Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224–230.
Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):77–83.
Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74–80.
Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):2267–2274.
Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):2091–2097.
Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):1238–1243.
Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422–432.
Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):18–26.
McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:68–72.
Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241–248.
Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224–230.
Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798–803.
Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301–323.
Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):1677–1679.
Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511–520.
Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126–136.
Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10.
Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):68–75.
Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569–576.
Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803–812.
Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317–324.
Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):1489–1495.
Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):1080–1085.
Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.

References

Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224–230.
Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):77–83.
Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:74–80.
Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):2506–2513.
Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):2267–2274.
Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):2091–2097.
Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):1238–1243.
Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422–432.
Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):18–26.
McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:68–72.
Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241–248.
Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224–230.
Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232–239.
Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719–724.
Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158–166.
van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798–803.
Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301–323.
Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):1677–1679.
Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511–520.
Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126–136.
Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10.
Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):68–75.
Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569–576.
Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803–812.
Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317–324.
Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):1489–1495.
Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):1080–1085.
Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.

Issue

Journal of Hospital Medicine - 8(1)

Issue

Journal of Hospital Medicine - 8(1)

Page Number

13-19

Page Number

13-19

Publications

Publications

Article Type

Display Headline

Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system

Display Headline

Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Division of Emergency Medicine, 300 Pasteur Drive, Alway Building, Room M121, Stanford, CA 94305

Content Gating

No Gating (article Unlocked/Free)

Alternative CME

Teaser Media

Article PDF Media

Detection of Physiologic Deterioration

Article Type

Changed

Mon, 05/22/2017 - 18:44

Display Headline

Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record

Author(s)

Patients in general medicalsurgical wards who experience unplanned transfer to the intensive care unit (ICU) have increased mortality and morbidity.13 Using an externally validated methodology permitting assessment of illness severity and mortality risk among all hospitalized patients,4, 5 we recently documented observed‐to‐expected mortality ratios >3.0 and excess length of stay of 10 days among patients who experienced such transfers.6

It is possible to predict adverse outcomes among monitored patients (eg, patients in the ICU or undergoing continuous electronic monitoring).7, 8 However, prediction of unplanned transfers among medicalsurgical ward patients presents challenges. Data collection (vital signs and laboratory tests) is relatively infrequent. The event rate (3% of hospital admissions) is low, and the rate in narrow time periods (eg, 12 hours) is extremely low: a hospital with 4000 admissions per year might experience 1 unplanned transfer to the ICU every 3 days. Not surprisingly, performance of models suitable for predicting ward patients' need for intensive care within narrow time frames have been disappointing.9 The Modified Early Warning Score (MEWS), has a c‐statistic, or area under the receiver operator characteristic of 0.67,1012 and our own model incorporating 14 laboratory tests, but no vital signs, has excellent performance with respect to predicting inpatient mortality, but poor performance with respect to unplanned transfer.6

In this report, we describe the development and validation of a complex predictive model suitable for use with ward patients. Our objective for this work was to develop a predictive model based on clinical and physiologic data available in real time from a comprehensive electronic medical record (EMR), not a clinically intuitive, manually assigned tool. The outcome of interest was unplanned transfer from the ward to the ICU, or death on the ward in a patient who was full code. This model has been developed as part of a regional effort to decrease preventable mortality in the Northern California Kaiser Permanente Medical Care Program (KPMCP), an integrated healthcare delivery system with 22 hospitals.

MATERIALS AND METHODS

For additional details, see the Supporting Information, Appendices 112, in the online version of this article.

This project was approved by the KPMCP Institutional Board for the Protection of Human Subjects.

The Northern California KPMCP serves a total population of approximately 3.3 million members. All Northern California KPMCP hospitals and clinics employ the same information systems with a common medical record number and can track care covered by the plan but delivered elsewhere. Databases maintained by the KPMCP capture admission and discharge times, admission and discharge diagnoses and procedures (assigned by professional coders), bed histories permitting quantification of intra‐hospital transfers, inter‐hospital transfers, as well as the results of all inpatient and outpatient laboratory tests. In July 2006, the KPMCP began deployment of the EMR developed by Epic Systems Corporation (www.epic. com), which has been adapted for the KPMCP and is known as KP HealthConnect (KPHC) in its hospitals. The last of these 22 hospitals went online in March 2010.

Our setting consisted of 14 hospitals in which the KPHC inpatient EMR had been running for at least 3 months (the KPMCP Antioch, Fremont, Hayward, Manteca, Modesto, Roseville, Sacramento, Santa Clara, San Francisco, Santa Rosa, South Sacramento, South San Francisco, Santa Teresa, and Walnut Creek hospitals). We have described the general characteristics of KPMCP hospitals elsewhere.4, 6 Our initial study population consisted of all patients admitted to these hospitals who met the following criteria: hospitalization began from November 1, 2006 through December 31, 2009; initial hospitalization occurred at a Northern California KPMCP hospital (ie, for inter‐hospital transfers, the first hospital stay occurred within the KPMCP); age 18 years; hospitalization was not for childbirth; and KPHC had been operational at the hospital for at least 3 months.

Analytic Approach

The primary outcome for this study was transfer to the ICU after admission to the hospital among patients residing either in a general medicalsurgical ward (ward) or transitional care unit (TCU), or death in the ward or TCU in a patient who was full code at the time of death (ie, had the patient survived, s/he would have been transferred to the ICU). The unit of analysis for this study was a 12‐hour patient shift, which could begin with a 7 _AM T₀ (henceforth, day shift) or a 7 _PM T₀ (night shift); in other words, we aimed to predict the occurrence of an event within 12 hours of T₀ using only data available prior to T₀. A shift in which a patient experienced the primary study outcome is an event shift, while one in which a patient did not experience the primary outcome is a comparison shift. Using this approach, an individual patient record could consist of both event and comparison shifts, since some patients might have multiple unplanned transfers and some patients might have none. Our basic analytic approach consisted of creating a cohort of event and comparison shifts (10 comparison shifts were randomly selected for each event shift), splitting the cohort into a derivation dataset (50%) and validation dataset (50%), developing a model using the derivation dataset, then applying the coefficients of the derivation dataset to the validation dataset. Because some event shifts were excluded due to the minimum 4‐hour length‐of‐stay requirement, we also applied model coefficients to these excluded shifts and a set of randomly selected comparison shifts.

Since the purpose of these analyses was to develop models with maximal signal extraction from sparsely collected predictors, we did not block a time period after the T₀ to allow for a reaction time to the alarm. Thus, since some events could occur immediately after the T₀ (as can be seen in the Supporting Information, Appendices, in the online version of this article), our models would need to be run at intervals that are more frequent than 2 times a day.

Independent Variables

In addition to patients' age and sex, we tested the following candidate independent variables. Some of these variables are part of the KPMCP risk adjustment model4, 5 and were available electronically for all patients in the cohort. We grouped admission diagnoses into 44 broad diagnostic categories (primary conditions), and admission types into 4 groups (emergency medical, emergency surgical, elective medical, and elective surgical). We quantified patients' degree of physiologic derangement in the 72 hours preceding hospitalization with a Laboratory‐based Acute Physiology Score (LAPS) using 14 laboratory test results prior to hospitalization; we also tested individual laboratory test results obtained after admission to the hospital. We quantified patients' comorbid illness burden using a COmorbidity Point Score (COPS) based on patients' preexisting diagnoses over the 12‐month period preceding hospitalization.4 We extracted temperature, heart rate, respiratory rate, systolic blood pressure, diastolic blood pressure, oxygen saturation, and neurological status from the EMR. We also tested the following variables based on specific information extracted from the EMR: shock index (heart rate divided by systolic blood pressure)13; care directive status (patients were placed into 4 groups: full code, partial code, do not resuscitate [DNR], and no care directive in place); and a proxy for measured lactate (PML; anion gap/serum bicarbonate 100).1416 For comparison purposes, we also created a retrospective electronically assigned MEWS, which we refer to as the MEWS(re), and we assigned this score to patient records electronically using data from KP HealthConnect.

Statistical Methods

Analyses were performed in SAS 9.1, Stata 10, and R 2.12. Final validation was performed using SAS (SAS Institute Inc., Carey, North Carolina). Since we did not limit ourselves to traditional severity‐scoring approaches (eg, selecting the worst heart rate in a given time interval), but also included trend terms (eg, change in heart rate over the 24 hours preceding T₀), the number of potential variables to test was very large. Detailed description of the statistical strategies employed for variable selection is provided in the Supporting Information, Appendices, in the online version of this article. Once variables were selected, our basic approach was to test a series of diagnosis‐specific logistic regression submodels using a variety of predictors that included vital signs, vital signs trends (eg, most recent heart rate minus earliest heart rate, heart rate over preceding 24 hours), and other above‐mentioned variables.

We assessed the ability of a submodel to correctly distinguish patients who died, from survivors, using the c‐statistic, as well as other metrics recommended by Cook.17 At the end of the modeling process, we pooled the results across all submodels. For vital signs, where the rate of missing data was <3%, we tested submodels in which we dropped shifts with missing data, as well as submodels in which we imputed missing vital signs to a normal value. For laboratory data, where the rate of missing data for a given shift was much greater, we employed a probabilistic imputation method that included consideration of when a laboratory test result became available.

RESULTS

During the study period, a total of 102,488 patients experienced 145,335 hospitalizations at the study hospitals. We removed 66 patients with 138 hospitalizations for data quality reasons, leaving us with our initial study sample of 102,422 patients whose characteristics are summarized in Table 1. Table 1, in which the unit of analysis is an individual patient, shows that patients who experienced the primary outcome were similar to those patients described in our previous report, in terms of their characteristics on admission as well as in experiencing excess morbidity and mortality.6

Table 1.Characteristics of Final Study Cohort
	Never Admitted to ICU	Direct Admit to ICU From ED	Unplanned Transfer to ICU*	Other ICU Admission
NOTE: All overnight admissions to the study hospitals excluding 66 patients who were removed due to incomplete data. Column categories are mutually exclusive and based on a patient's first hospitalization during the study time period. Abbreviations: COPS, COmorbidity Point Score, DNR, do not resuscitate; ED, emergency department; GI, gastrointestinal; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; SD, standard deviation. * This group consists of all patients who meet our case definition and includes: 1) patients who had an unplanned transfer to the ICU from the transitional care unit (TCU) or ward; and 2) patients who died on the ward without a DNR order in place at the time of death (ie, who would have been transferred to the ICU had they survived). This group includes patients admitted directly to the ICU from the operating room, post‐anesthesia recovery, or an unknown unit, as well as patients with a planned transfer to the ICU. LAPS point score based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and inpatient mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%; <7 to 30 with a mortality risk of 1%5%; 30 to 60 with a mortality risk of 5%9%; and >60 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. COPS point score based on a patient's healthcare utilization diagnoses (during the year preceding admission to the hospital). Analogous to present on admission (POA) coding. Scores can range from 0 to a theoretical maximum of 701, but scores >200 are rare. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and inpatient mortality is as follows: a COPS <50 is associated with a mortality risk of <1%; <100 with a mortality risk of 1%5%; 100 to 145 with a mortality risk of 5%10%; and >145 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. ∥Numbers for patients who survived last hospitalization to discharge are available upon request.
N	89,269	5963	2880	4310
Age (mean SD)	61.26 18.62	62.25 18.13	66.12 16.20	64.45 15.91
Male (n, %)	37,228 (41.70%)	3091 (51.84%)	1416 (49.17%)	2378 (55.17%)
LAPS (mean SD)	13.02 15.79	32.72 24.85	24.83 21.53	11.79 18.16
COPS(mean SD)	67.25 51.42	73.88 57.42	86.33 59.33	78.44 52.49
% Predicted mortality risk (mean SD)	1.93% 3.98%	7.69% 12.59%	5.23% 7.70%	3.66% 6.81%
Survived first hospitalization to discharge^∥	88,479 (99.12%)	5336 (89.49%)	2316 (80.42%)	4063 (94.27%)
Care order on admission
Full code	78,877 (88.36%)	5198 (87.17%)	2598 (90.21%)	4097 (95.06%)
Partial code	664 (0.74%)	156 (2.62%)	50 (1.74%)	27 (0.63%)
Comfort care	21 (0.02%)	2 (0.03%)	0 (0%)	0 (0%)
DNR	8227 (9.22%)	539 (9.04%)	219 (7.60%)	161 (3.74%)
Comfort care and DNR	229 (0.26%)	9 (0.15%)	2 (0.07%)	2 (0.05%)
No order	1251 (1.40%)	59 (0.99%)	11 (0.38%)	23 (0.53%)
Admission diagnosis (n, %)
Pneumonia	2385 (2.67%)	258 (4.33%)	242 (8.40%)	68 (1.58%)
Sepsis	5822 (6.52%)	503 (8.44%)	279 (9.69%)	169 (3.92%)
GI bleeding	9938 (11.13%)	616 (10.33%)	333 (11.56%)	290 (6.73%)
Cancer	2845 (3.19%)	14 (0.23%)	95 (3.30%)	492 (11.42%)
Total hospital length of stay (days SD)	3.08 3.29	5.37 7.50	12.16 13.12	8.06 9.53

Figure 1shows how we developed the analysis cohort, by removing patients with a comfort‐care‐only order placed within 4 hours after admission (369 patients/744 hospitalizations) and patients who were never admitted to the ward or TCU (7,220/10,574). This left a cohort consisting of 94,833 patients who experienced 133,879 hospitalizations spanning a total of 1,079,062 shifts. We then removed shifts where: 1) a patient was not on the ward at the start of a shift, or was on the ward for <4 hours of a shift; 2) the patient had a comfort‐care order in place at the start of the shift; and 3) the patient died and was ineligible to be a case (the patient had a DNR order in place or died in the ICU). The final cohort eligible for sampling consisted of 846,907 shifts, which involved a total of 92,797 patients and 130,627 hospitalizations. There were a total of 4,036 event shifts, which included 3,224 where a patient was transferred from the ward to the ICU, 717 from the TCU to the ICU, and 95 where a patient died on the ward or TCU without a DNR order in place. We then randomly selected 39,782 comparison shifts. Thus, our final cohort for analysis included 4,036 event shifts (1,979 derivation/2,057 validation and 39,782 comparison shifts (19,509/20,273). As a secondary validation, we also applied model coefficients to the 429 event shifts excluded due to the <4‐hour length‐of‐stay requirement.

Development of sampling cohort. *There are 429 event shifts excluded; see text for details. Abbreviations: DNR, do not resuscitate; ICU, intensive care unit; TCU, transitional care unit.

Table 2 compares event shifts with comparison shifts. In the 24 hours preceding ICU transfer, patients who were subsequently transferred had statistically significant, but not necessarily clinically significant, differences in terms of these variables. However, missing laboratory data were more common, ranging from 18% to 31% of all shifts (we did not incorporate laboratory tests where 35% of the shifts had missing data for that test).

Table 2.Event and Comparison Shifts
Predictor	Event Shifts	Comparison Shifts	P
NOTE: Code status, vital sign, and laboratory values measures closest to the start of the shift (7 _AM or 7 _PM) are used. Abbreviations: COPS, COmorbidity Point Score; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; MEWS(re), Modified Early Warning Score (retrospective electronic); SD, standard deviation. * LAPS; see Table 1, text, and Escobar et al4 for more details. COPS; see Table 1, text, and Escobar et al4 for more details. Refers to patients who had an active full code order at the start of the sampling time frame. See text for explanation of sampling time frame, and how both cases and controls could have been in the ICU. ∥ See text for explanation of how both cases and controls could have experienced an unplanned transfer to the ICU. MEWS(re); see text and Subbe et al10 for a description of this score. # (Anion gap bicarbonate) 100. ** Rates of missing data for vital signs are not shown because <3% of the shifts were missing these data.
Number	4036	39,782
Age (mean SD)	67.19 15.25	65.41 17.40	<0.001
Male (n, %)	2007 (49.73%)	17,709 (44.52%)	<0.001
Day shift	1364 (33.80%)	17,714 (44.53%)	<0.001
LAPS*	27.89 22.10	20.49 20.16	<0.001
COPS	116.33 72.31	100.81 68.44	<0.001
Full code (n, %)	3496 (86.2%)	32,156 (80.8%)	<0.001
ICU shift during hospitalization	3964 (98.22%)	7197 (18.09%)	<0.001
Unplanned transfer to ICU during hospitalization∥	353 (8.8%)	1466 (3.7%)	<0.001
Temperature (mean SD)	98.15 (1.13)	98.10 (0.85)	0.009
Heart rate (mean SD)	90.30 (20.48)	79.86 (5.27)	<0.001
Respiratory rate (mean SD)	20.36 (3.70)	18.87 (1.79)	<0.001
Systolic blood pressure (mean SD)	123.65 (23.26)	126.21 (19.88)	<0.001
Diastolic blood pressure (mean SD)	68.38 (14.49)	69.46 (11.95)	<0.001
Oxygen saturation (mean SD)	95.72% (3.00)	96.47 % (2.26)	<0.001
MEWS(re) (mean SD)	3.64 (2.02)	2.34 (1.61)	<0.001
% <5	74.86%	92.79%
% 5	25.14%	7.21%	<0.001
Proxy for measured lactate# (mean SD)	36.85 (28.24)	28.73 (16.74)	<0.001
% Missing in 24 hr before start of shift**	17.91%	28.78%	<0.001
Blood urea nitrogen (mean SD)	32.03 (25.39)	22.72 (18.9)	<0.001
% Missing in 24 hr before start of shift	19.67%	20.90%	<0.001
White blood cell count 1000 (mean SD)	12.33 (11.42)	9.83 (6.58)	<0.001
% Missing in 24 hr before start of shift	21.43%	30.98%	<0.001
Hematocrit (mean SD)	33.08 (6.28)	33.07 (5.25)	0.978
% Missing in 24 hr before start of shift	19.87%	29.55%	<0.001

After conducting multiple analyses using the derivation dataset, we developed 24 submodels, a compromise between our finding that primary‐condition‐specific models showed better performance and the fact that we had very few events among patients with certain primary conditions (eg, pericarditis/valvular heart disease), which forced us to create composite categories (eg, a category pooling patients with pericarditis, atherosclerosis, and peripheral vascular disease). Table 3 lists variables included in our final submodels.

Table 3.Variables Included in Final Electronic Medical Record‐Based Models
Variable	Description
Abbreviations: COPS, COmorbidity Point Score; LAPS, Laboratory Acute Physiology Score; LOS, length of stay. * LAPS based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. See text and Escobar et al4 for details. COPS based on a patient's diagnoses in the 12 mo preceding hospitalization. See text and Escobar et al4 for details. Indicator variable (for patients in whom a COPS could not be obtained) also included in models. See text and Supporting Information, Appendices, in the online version of this article for details on imputation strategy employed when values were missing. See Wrenn14 and Rocktaeschel et al16 for justification for use of the combination of anion gap and serum bicarbonate.
Directive status	Full code or not full code
LAPS*	Admission physiologic severity of illness score (continuous variable ranging from 0 to 256). Standardized and included as LAPS and LAPS squared
COPS	Comorbidity burden score (continuous variable ranging from 0 to 701). Standardized and included as COPS and COPS squared.
COPS status	Indicator for absent comorbidity data
LOS at T₀	Length of stay in the hospital (total time in hours) at the T₀; standardized.
T₀ time of day	7 _AM or 7 _PM
Temperature	Worst (highest) temperature in 24 hr preceding T₀; variability in temperature in 24 hr preceding T₀.
Heart rate	Most recent heart rate in 24 hr preceding T₀; variability in heart rate in 24 hr preceding T₀.
Respiratory rate	Most recent respiratory rate in 24 hr preceding T₀; worst (highest) respiratory rate in 24 hr preceding T₀; variability in respiratory rate in 24 hr preceding T₀.
Diastolic blood pressure	Most recent diastolic blood pressure in 24 hr preceding T₀ transformed by subtracting 70 from the actual value and squaring the result. Any value above 2000 is subsequently then set to 2000, yielding a continuous variable ranging from 0 to 2000.
Systolic pressure	Variability in systolic blood pressure in 24 hr preceding T₀.

Pulse oximetry	Worst (lowest) oxygen saturation in 24 hr preceding T₀; variability in oxygen saturation in 24 hr preceding T₀.
Neurological status	Most recent neurological status check in 24 hr preceding T₀.
Laboratory tests	Blood urea nitrogen
	Proxy for measured lactate = (anion gap serum bicarbonate) 100
	Hematocrit
	Total white blood cell count

Table 4 summarizes key results in the validation dataset. Across all diagnoses, the MEWS(re) had c‐statistic of 0.709 (95% confidence interval, 0.6970.721) in the derivation dataset and 0.698 (0.6860.710) in the validation dataset. In the validation dataset, the MEWS(re) performed best among patients with a set of gastrointestinal diagnoses (c = 0.792; 0.7260.857) and worst among patients with congestive heart failure (0.541; 0.5000.620). In contrast, across all primary conditions, the EMR‐based models had a c‐statistic of 0.845 (0.8260.863) in the derivation dataset and 0.775 (0.7530.797) in the validation dataset. In the validation dataset, the EMR‐based models also performed best among patients with a set of gastrointestinal diagnoses (0.841; 0.7830.897) and worst among patients with congestive heart failure (0.683; 0.6100.755). A negative correlation (R = 0.63) was evident between the number of event shifts in a submodel and the drop in the c‐statistic seen in the validation dataset.

Table 4.Best and Worst Performing Submodels in the Validation Dataset
	No. of Shifts in Validation Dataset		c‐Statistic
Diagnoses Group*	Event	Comparison	MEWS(re)	EMR Model
Abbreviations: EMR, electronic medical record; GI, gastrointestinal; MEWS(re), Modified Early Warning Score (retrospective electronic). * Specific International Classification of Diseases (ICD) codes used are detailed in the Supporting Information, Appendices, in the online version of this article. MEWS(re); see text, Supporting Information, Appendices, in the online version of this article, and Subbe et al10 for more details. Model based on comprehensive data from EMR; see text, Table 3, and Supporting Information, Appendices, in the online version of this article for more details. This group of diagnoses includes appendicitis, cholecystitis, cholangitis, hernias, and pancreatic disorders. ∥ This group of diagnoses includes: gastrointestinal hemorrhage, miscellaneous disorders affecting the stomach and duodenum, diverticulitis, abdominal symptoms, nausea with vomiting, and blood in stool. This group of diagnoses includes inflammatory bowel disease, malabsorption syndromes, gastrointestinal obstruction, and enteritides.
Acute myocardial infarction	36	169	0.541	0.572
Diseases of pulmonary circulation and cardiac dysrhythmias	40	329	0.565	0.645
Seizure disorders	45	497	0.594	0.647
Rule out myocardial infarction	77	727	0.602	0.648
Pneumonia	163	847	0.741	0.801
GI diagnoses, set A	58	942	0.755	0.803
GI diagnoses, set B∥	256	2,610	0.772	0.806
GI diagnoses, set C	46	520	0.792	0.841
All diagnosis	2,032	20,106	0.698	0.775

We also compared model performance when our datasets were restricted to 1 randomly selected observation per patient; in these analyses, the total number of event shifts was 3,647 and the number of comparison shifts was 29,052. The c‐statistic for the MEWS(re) in the derivation dataset was 0.709 (0.6940.725); in the validation dataset, it was 0.698 (0.6920.714). The corresponding values for the EMR‐based models were 0.856 (0.8350.877) and 0.780 (0.7560.804). We also tested models in which, instead of dropping shifts with missing vital signs, we imputed missing vital signs to their normal value. The c‐statistic for the EMR‐based model with imputed vital sign values was 0.842 (0.8230.861) in the derivation dataset and 0.773 (0.7520.794) in the validation dataset. Lastly, we applied model coefficients to a dataset consisting of 4,290 randomly selected comparison shifts plus the 429 shifts excluded because of the 4‐hour length‐of‐stay criterion. The c‐statistic for this analysis was 0.756 (0.7030.809).

As a general rule, the EMR‐based models were more than twice as efficient as the MEWS(re). For example, a MEWS(re) threshold of 6 as the trigger for an alarm would identify 15% of all transfers to the ICU, with 34.4 false alarms for each transfer; in contrast, using the EMR‐based approach to identify 15% of all transfers, there were 14.5 false alarms for each transfer. Applied to the entire KPMCP Northern California Region, using the MEWS(re), a total of 52 patients per day would need to be evaluated, but only 22 per day using the EMR‐based approach. If one employed a MEWS(re) threshold of 4, this would lead to identification of 44% of all transfers, with a ratio of 69 false alarms for each transfer; using the EMR, the ratio would be 34 to 1. Across the entire KPMCP, a total of 276 patients per day (or about 19.5 a day per hospital) would need to be evaluated using the MEWS(re), but only 136 (or about 9.5 per hospital per day) using the EMR.

DISCUSSION

Using data from a large hospital cohort, we have developed a predictive model suitable for use in non‐ICU populations cared for in integrated healthcare settings with fully automated EMRs. The overall performance of our model, which incorporates acute physiology, diagnosis, and longitudinal data, is superior to the predictive ability of a model that can be assigned manually. This is not surprising, given that scoring systems such as the MEWS make an explicit tradeoff losing information found in multiple variables in exchange for ease of manual assignment. Currently, the model described in this report is being implemented in a simulated environment, a final safety test prior to piloting real‐time provision of probability estimates to clinicians and nurses. Though not yet ready for real‐time use, it is reasonable for our model to be tested using the KPHC shadow server, since evaluation in a simulated environment constitutes a critical evaluation step prior to deployment for clinical use. We also anticipate further refinement and revalidation to occur as more inpatient data become available in the KPMCP and elsewhere.

A number of limitations to our approach must be emphasized. In developing our models, we determined that, while modeling by clinical condition was important, the study outcome was rare for some primary conditions. In these diagnostic groups, which accounted for 12.5% of the event shifts and 10.6% of the comparison shifts, the c‐statistic in the validation dataset was <0.70. Since all 22 KPMCP hospitals are now online and will generate an additional 150,000 adult hospitalizations per year, we expect to be able to correct this problem prior to deployment of these models for clinical use. Having additional data will permit us to improve model discrimination and thus decrease the evaluation‐to‐detection ratio. In future iterations of these models, more experimentation with grouping of International Classification of Diseases (ICD) codes may be required. The problem of grouping ICD codes is not an easy one to resolve, in that diagnoses in the grouping must share common pathophysiology while having a grouping with a sufficient number of adverse events for stable statistical models.

Ideally, it would have been desirable to employ a more objective measure of deterioration, since the decision to transfer a patient to the ICU is discretionary. However, we have found that key data points needed to define such a measure (eg, vital signs) are not consistently charted when a patient deterioratesthis is not surprising outside the research setting, given that nurses and physicians involved in a transfer may be focusing on caring for the patient rather than immediately charting. Given the complexities of end‐of‐life‐care decision‐making, we could not employ death as the outcome of interest. A related issue is that our model does not differentiate between reasons for needing transfer to the ICU, an issue recently discussed by Bapoje et al.18

Our model does not address an important issue raised by Bapoje et al18 and Litvak, Pronovost, and others,19, 20 namely, whether a patient should have been admitted to a non‐ICU setting in the first place. Our team is currently developing a model for doing exactly this (providing decision support for triage in the emergency department), but discussion of this methodology is outside the scope of this article.

Because of resource and data limitations, our model also does not include newborns, children, women admitted for childbirth, or patients transferred from non‐KPMCP hospitals. However, the approach described here could serve as a starting point for developing models for these other populations.

The generalizability of our model must also be considered. The Northern California KPMCP is unusual in having large electronic databases that include physiologic as well as longitudinal patient data. Many hospitals cannot take advantage of all the methods described here. However, the methods we employed could be modified for use by hospital systems in countries such as Great Britain and Canada, and entities such as the Veterans Administration Hospital System in the United States. The KPMCP population, an insured population with few barriers to access, is healthier than the general population, and some population subsets are underrepresented in our cohort. Practice patterns may also vary. Nonetheless, the model described here could serve as a good starting point for future collaborative studies, and it would be possible to develop models suitable for use by stand‐alone hospitals (eg, recalibrating so that one used a Charlson comorbidity21 score based on present on‐admission codes rather than the COPS).

The need for early detection of patient deterioration has played a major role in the development of rapid response teams, as well as scores such as the MEWS. In particular, entities such as the Institute for Healthcare Improvement have advocated the use of early warning systems.22 However, having a statistically robust model to support an early warning system is only part of the solution, and a number of new challenges must then be addressed. The first is actual electronic deployment. Existing inpatient EMRs were not designed with complex calculations in mind, and we anticipate that some degradation in performance will occur when we test our models using real‐time data capture. As Bapoje et al point out, simply having an alert may be insufficient, since not all transfers are preventable.18 Early warning systems also raise ethical issues (for example, what should be done if an alert leads a clinician to confront the fact that an end‐of‐life‐care discussion needs to occur?). From a research perspective, if one were to formally test the benefits of such models, it would be critical to define outcome measures other than death (which is strongly affected by end‐of‐life‐care decisions) or ICU transfer (which is often desirable).

In conclusion, we have developed an approach for predicting impending physiologic deterioration of hospitalized adults outside the ICU. Our approach illustrates how organizations can take maximal advantage of EMRs in a manner that exceeds meaningful use specifications.23, 24 Our study highlights the possibility of using fully automated EMR data for building and applying sophisticated statistical models in settings other than the highly monitored ICU without the need for additional equipment. It also expands the universe of severity scoring to one in which probability estimates are provided in real time and throughout an entire hospitalization. Model performance will undoubtedly improve over time, as more patient data become available. Although our approach has important limitations, it is suitable for testing using real‐time data in a simulated environment. Such testing would permit identification of unanticipated problems and quantification of the degradation of model performance due to real life factors, such as delays in vital signs charting or EMR system brownouts. It could also serve as the springboard for future collaborative studies, with a broader population base, in which the EMR becomes a tool for care, not just documentation.

Acknowledgements

We thank Ms Marla Gardner and Mr John Greene for their work in the development phase of this project. We are grateful to Brian Hoberman, Andrew Hwang, and Marc Flagg from the RIMS group; to Colin Stobbs, Sriram Thiruvenkatachari, and Sundeep Sood from KP IT, Inc; and to Dennis Andaya, Linda Gliner, and Cyndi Vasallo for their assistance with data‐quality audits. We are also grateful to Dr Philip Madvig, Dr Paul Feigenbaum, Dr Alan Whippy, Mr Gregory Adams, Ms Barbara Crawford, and Dr Marybeth Sharpe for their administrative support and encouragement; and to Dr Alan S. Go, Acting Director of the Kaiser Permanente Division of Research, for reviewing the manuscript.

Files

References

Barnett MJ,Kaboli PJ,Sirio CA,Rosenthal GE.Day of the week of intensive care admission and patient outcomes: a multisite regional evaluation.Med Care.2002;40(6):530–539.
Ensminger SA,Morales IJ,Peters SG, et al.The hospital mortality of patients admitted to the ICU on weekends.Chest.2004;126(4):1292–1298.
Luyt CE,Combes A,Aegerter P, et al.Mortality among patients admitted to intensive care units during weekday day shifts compared with “off” hours.Crit Care Med.2007;35(1):3–11.
Escobar G,Greene J,Scheirer P,Gardner M,Draper D,Kipnis P.Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases.Med Care.2008;46(3):232–239.
van Walraven C,Escobar GJ,Greene JD,Forster AJ.The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population.J Clin Epidemiol.2010;63(7):798–803.
Escobar GJ,Greene JD,Gardner MN,Marelich GP,Quick B,Kipnis P.Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS).J Hosp Med.2011;6(2):74–80.
Chambrin MC,Ravaux P,Calvelo‐Aros D,Jaborska A,Chopin C,Boniface B.Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis.Intensive Care Med.1999;25(12):1360–1366.
Saria S,Rajani AK,Gould J,Koller D,Penn AA.Integration of early physiological responses predicts later illness severity in preterm infants.Sci Transl Med.2010;2(48):48ra65.
Subbe CP,Gao H,Harrison DA.Reproducibility of physiological track‐and‐trigger warning systems for identifying at‐risk patients on the ward.Intensive Care Med.2007;33(4):619–624.
Subbe CP,Kruger M,Rutherford P,Gemmel L.Validation of a Modified Early Warning Score in medical admissions.Q J Med.2001;94:521–526.
Subbe CP,Davies RG,Williams E,Rutherford P,Gemmell L.Effect of introducing the Modified Early Warning score on clinical outcomes, cardio‐pulmonary arrests and intensive care utilisation in acute medical admissions.Anaesthesia.2003;58(8):797–802.
MERIT Study Investigators.Introduction of the medical emergency team (MET) system: a cluster‐randomized controlled trial.Lancet.2005;365(9477):2091–2097.
Keller AS,Kirkland LL,Rajasekaran SY,Cha S,Rady MY,Huddleston JM.Unplanned transfers to the intensive care unit: the role of the shock index.J Hosp Med.2010;5(8):460–465.
Wrenn K.The delta (delta) gap: an approach to mixed acid‐base disorders.Ann Emerg Med.1990;19(11):1310–1313.
Williamson JC.Acid‐base disorders: classification and management strategies.Am Fam Physician.1995;52(2):584–590.
Rocktaeschel J,Morimatsu H,Uchino S,Bellomo R.Unmeasured anions in critically ill patients: can they predict mortality?Crit Care Med.2003;31(8):2131–2136.
Cook NR.Use and misuse of the receiver operating characteristic curve in risk prediction.Circulation.2007;115(7):928–935.
Bapoje SR,Gaudiani JL,Narayanan V,Albert RK.Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care.J Hosp Med.2011;6(2):68–72.
Litvak E,Pronovost PJ.Rethinking rapid response teams.JAMA.2010;304(12):1375–1376.
Winters BD,Pham J,Pronovost PJ.Rapid response teams—walk, don't run.JAMA.2006;296(13):1645–1647.
Charlson ME,Pompei P,Ales KL,MacKenzie CR.A new method of classifying prognostic comorbidity in longitudinal populations: development and validation.J Chronic Dis.1987;40:373–383.
Institute for Healthcare Improvement.Early Warning Systems:The Next Level of Rapid Response.2011. http://www.ihi.org/IHI/Programs/AudioAndWebPrograms/ExpeditionEarlyWarningSystemsTheNextLevelofRapidResponse.htm?player=wmp. Accessed 4/6/11.
Bowes WA.Assessing readiness for meeting meaningful use: identifying electronic health record functionality and measuring levels of adoption.AMIA Annu Symp Proc.2010;2010:66–70.
Medicare and Medicaid Programs;Electronic Health Record Incentive Program. Final Rule.Fed Reg.2010;75(144):44313–44588.

Article PDF

Issue

Journal of Hospital Medicine - 7(5)

Publications

Page Number

388-395

Read more about Detection of Physiologic Deterioration

Sections

Author(s)

Author(s)

Files

Files

Article PDF

Article PDF

MATERIALS AND METHODS

For additional details, see the Supporting Information, Appendices 112, in the online version of this article.

This project was approved by the KPMCP Institutional Board for the Protection of Human Subjects.

Analytic Approach

Independent Variables

Statistical Methods

RESULTS

Table 1.Characteristics of Final Study Cohort
	Never Admitted to ICU	Direct Admit to ICU From ED	Unplanned Transfer to ICU*	Other ICU Admission
NOTE: All overnight admissions to the study hospitals excluding 66 patients who were removed due to incomplete data. Column categories are mutually exclusive and based on a patient's first hospitalization during the study time period. Abbreviations: COPS, COmorbidity Point Score, DNR, do not resuscitate; ED, emergency department; GI, gastrointestinal; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; SD, standard deviation. * This group consists of all patients who meet our case definition and includes: 1) patients who had an unplanned transfer to the ICU from the transitional care unit (TCU) or ward; and 2) patients who died on the ward without a DNR order in place at the time of death (ie, who would have been transferred to the ICU had they survived). This group includes patients admitted directly to the ICU from the operating room, post‐anesthesia recovery, or an unknown unit, as well as patients with a planned transfer to the ICU. LAPS point score based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and inpatient mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%; <7 to 30 with a mortality risk of 1%5%; 30 to 60 with a mortality risk of 5%9%; and >60 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. COPS point score based on a patient's healthcare utilization diagnoses (during the year preceding admission to the hospital). Analogous to present on admission (POA) coding. Scores can range from 0 to a theoretical maximum of 701, but scores >200 are rare. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and inpatient mortality is as follows: a COPS <50 is associated with a mortality risk of <1%; <100 with a mortality risk of 1%5%; 100 to 145 with a mortality risk of 5%10%; and >145 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. ∥Numbers for patients who survived last hospitalization to discharge are available upon request.
N	89,269	5963	2880	4310
Age (mean SD)	61.26 18.62	62.25 18.13	66.12 16.20	64.45 15.91
Male (n, %)	37,228 (41.70%)	3091 (51.84%)	1416 (49.17%)	2378 (55.17%)
LAPS (mean SD)	13.02 15.79	32.72 24.85	24.83 21.53	11.79 18.16
COPS(mean SD)	67.25 51.42	73.88 57.42	86.33 59.33	78.44 52.49
% Predicted mortality risk (mean SD)	1.93% 3.98%	7.69% 12.59%	5.23% 7.70%	3.66% 6.81%
Survived first hospitalization to discharge^∥	88,479 (99.12%)	5336 (89.49%)	2316 (80.42%)	4063 (94.27%)
Care order on admission
Full code	78,877 (88.36%)	5198 (87.17%)	2598 (90.21%)	4097 (95.06%)
Partial code	664 (0.74%)	156 (2.62%)	50 (1.74%)	27 (0.63%)
Comfort care	21 (0.02%)	2 (0.03%)	0 (0%)	0 (0%)
DNR	8227 (9.22%)	539 (9.04%)	219 (7.60%)	161 (3.74%)
Comfort care and DNR	229 (0.26%)	9 (0.15%)	2 (0.07%)	2 (0.05%)
No order	1251 (1.40%)	59 (0.99%)	11 (0.38%)	23 (0.53%)
Admission diagnosis (n, %)
Pneumonia	2385 (2.67%)	258 (4.33%)	242 (8.40%)	68 (1.58%)
Sepsis	5822 (6.52%)	503 (8.44%)	279 (9.69%)	169 (3.92%)
GI bleeding	9938 (11.13%)	616 (10.33%)	333 (11.56%)	290 (6.73%)
Cancer	2845 (3.19%)	14 (0.23%)	95 (3.30%)	492 (11.42%)
Total hospital length of stay (days SD)	3.08 3.29	5.37 7.50	12.16 13.12	8.06 9.53

Table 2.Event and Comparison Shifts
Predictor	Event Shifts	Comparison Shifts	P
NOTE: Code status, vital sign, and laboratory values measures closest to the start of the shift (7 _AM or 7 _PM) are used. Abbreviations: COPS, COmorbidity Point Score; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; MEWS(re), Modified Early Warning Score (retrospective electronic); SD, standard deviation. * LAPS; see Table 1, text, and Escobar et al4 for more details. COPS; see Table 1, text, and Escobar et al4 for more details. Refers to patients who had an active full code order at the start of the sampling time frame. See text for explanation of sampling time frame, and how both cases and controls could have been in the ICU. ∥ See text for explanation of how both cases and controls could have experienced an unplanned transfer to the ICU. MEWS(re); see text and Subbe et al10 for a description of this score. # (Anion gap bicarbonate) 100. ** Rates of missing data for vital signs are not shown because <3% of the shifts were missing these data.
Number	4036	39,782
Age (mean SD)	67.19 15.25	65.41 17.40	<0.001
Male (n, %)	2007 (49.73%)	17,709 (44.52%)	<0.001
Day shift	1364 (33.80%)	17,714 (44.53%)	<0.001
LAPS*	27.89 22.10	20.49 20.16	<0.001
COPS	116.33 72.31	100.81 68.44	<0.001
Full code (n, %)	3496 (86.2%)	32,156 (80.8%)	<0.001
ICU shift during hospitalization	3964 (98.22%)	7197 (18.09%)	<0.001
Unplanned transfer to ICU during hospitalization∥	353 (8.8%)	1466 (3.7%)	<0.001
Temperature (mean SD)	98.15 (1.13)	98.10 (0.85)	0.009
Heart rate (mean SD)	90.30 (20.48)	79.86 (5.27)	<0.001
Respiratory rate (mean SD)	20.36 (3.70)	18.87 (1.79)	<0.001
Systolic blood pressure (mean SD)	123.65 (23.26)	126.21 (19.88)	<0.001
Diastolic blood pressure (mean SD)	68.38 (14.49)	69.46 (11.95)	<0.001
Oxygen saturation (mean SD)	95.72% (3.00)	96.47 % (2.26)	<0.001
MEWS(re) (mean SD)	3.64 (2.02)	2.34 (1.61)	<0.001
% <5	74.86%	92.79%
% 5	25.14%	7.21%	<0.001
Proxy for measured lactate# (mean SD)	36.85 (28.24)	28.73 (16.74)	<0.001
% Missing in 24 hr before start of shift**	17.91%	28.78%	<0.001
Blood urea nitrogen (mean SD)	32.03 (25.39)	22.72 (18.9)	<0.001
% Missing in 24 hr before start of shift	19.67%	20.90%	<0.001
White blood cell count 1000 (mean SD)	12.33 (11.42)	9.83 (6.58)	<0.001
% Missing in 24 hr before start of shift	21.43%	30.98%	<0.001
Hematocrit (mean SD)	33.08 (6.28)	33.07 (5.25)	0.978
% Missing in 24 hr before start of shift	19.87%	29.55%	<0.001

Table 3.Variables Included in Final Electronic Medical Record‐Based Models
Variable	Description
Abbreviations: COPS, COmorbidity Point Score; LAPS, Laboratory Acute Physiology Score; LOS, length of stay. * LAPS based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. See text and Escobar et al4 for details. COPS based on a patient's diagnoses in the 12 mo preceding hospitalization. See text and Escobar et al4 for details. Indicator variable (for patients in whom a COPS could not be obtained) also included in models. See text and Supporting Information, Appendices, in the online version of this article for details on imputation strategy employed when values were missing. See Wrenn14 and Rocktaeschel et al16 for justification for use of the combination of anion gap and serum bicarbonate.
Directive status	Full code or not full code
LAPS*	Admission physiologic severity of illness score (continuous variable ranging from 0 to 256). Standardized and included as LAPS and LAPS squared
COPS	Comorbidity burden score (continuous variable ranging from 0 to 701). Standardized and included as COPS and COPS squared.
COPS status	Indicator for absent comorbidity data
LOS at T₀	Length of stay in the hospital (total time in hours) at the T₀; standardized.
T₀ time of day	7 _AM or 7 _PM
Temperature	Worst (highest) temperature in 24 hr preceding T₀; variability in temperature in 24 hr preceding T₀.
Heart rate	Most recent heart rate in 24 hr preceding T₀; variability in heart rate in 24 hr preceding T₀.
Respiratory rate	Most recent respiratory rate in 24 hr preceding T₀; worst (highest) respiratory rate in 24 hr preceding T₀; variability in respiratory rate in 24 hr preceding T₀.
Diastolic blood pressure	Most recent diastolic blood pressure in 24 hr preceding T₀ transformed by subtracting 70 from the actual value and squaring the result. Any value above 2000 is subsequently then set to 2000, yielding a continuous variable ranging from 0 to 2000.
Systolic pressure	Variability in systolic blood pressure in 24 hr preceding T₀.

Pulse oximetry	Worst (lowest) oxygen saturation in 24 hr preceding T₀; variability in oxygen saturation in 24 hr preceding T₀.
Neurological status	Most recent neurological status check in 24 hr preceding T₀.
Laboratory tests	Blood urea nitrogen
	Proxy for measured lactate = (anion gap serum bicarbonate) 100
	Hematocrit
	Total white blood cell count

Table 4.Best and Worst Performing Submodels in the Validation Dataset
	No. of Shifts in Validation Dataset		c‐Statistic
Diagnoses Group*	Event	Comparison	MEWS(re)	EMR Model
Abbreviations: EMR, electronic medical record; GI, gastrointestinal; MEWS(re), Modified Early Warning Score (retrospective electronic). * Specific International Classification of Diseases (ICD) codes used are detailed in the Supporting Information, Appendices, in the online version of this article. MEWS(re); see text, Supporting Information, Appendices, in the online version of this article, and Subbe et al10 for more details. Model based on comprehensive data from EMR; see text, Table 3, and Supporting Information, Appendices, in the online version of this article for more details. This group of diagnoses includes appendicitis, cholecystitis, cholangitis, hernias, and pancreatic disorders. ∥ This group of diagnoses includes: gastrointestinal hemorrhage, miscellaneous disorders affecting the stomach and duodenum, diverticulitis, abdominal symptoms, nausea with vomiting, and blood in stool. This group of diagnoses includes inflammatory bowel disease, malabsorption syndromes, gastrointestinal obstruction, and enteritides.
Acute myocardial infarction	36	169	0.541	0.572
Diseases of pulmonary circulation and cardiac dysrhythmias	40	329	0.565	0.645
Seizure disorders	45	497	0.594	0.647
Rule out myocardial infarction	77	727	0.602	0.648
Pneumonia	163	847	0.741	0.801
GI diagnoses, set A	58	942	0.755	0.803
GI diagnoses, set B∥	256	2,610	0.772	0.806
GI diagnoses, set C	46	520	0.792	0.841
All diagnosis	2,032	20,106	0.698	0.775

DISCUSSION

Acknowledgements

MATERIALS AND METHODS

For additional details, see the Supporting Information, Appendices 112, in the online version of this article.

This project was approved by the KPMCP Institutional Board for the Protection of Human Subjects.

Analytic Approach

Independent Variables

Statistical Methods

RESULTS

Table 1.Characteristics of Final Study Cohort
	Never Admitted to ICU	Direct Admit to ICU From ED	Unplanned Transfer to ICU*	Other ICU Admission
NOTE: All overnight admissions to the study hospitals excluding 66 patients who were removed due to incomplete data. Column categories are mutually exclusive and based on a patient's first hospitalization during the study time period. Abbreviations: COPS, COmorbidity Point Score, DNR, do not resuscitate; ED, emergency department; GI, gastrointestinal; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; SD, standard deviation. * This group consists of all patients who meet our case definition and includes: 1) patients who had an unplanned transfer to the ICU from the transitional care unit (TCU) or ward; and 2) patients who died on the ward without a DNR order in place at the time of death (ie, who would have been transferred to the ICU had they survived). This group includes patients admitted directly to the ICU from the operating room, post‐anesthesia recovery, or an unknown unit, as well as patients with a planned transfer to the ICU. LAPS point score based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and inpatient mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%; <7 to 30 with a mortality risk of 1%5%; 30 to 60 with a mortality risk of 5%9%; and >60 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. COPS point score based on a patient's healthcare utilization diagnoses (during the year preceding admission to the hospital). Analogous to present on admission (POA) coding. Scores can range from 0 to a theoretical maximum of 701, but scores >200 are rare. With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and inpatient mortality is as follows: a COPS <50 is associated with a mortality risk of <1%; <100 with a mortality risk of 1%5%; 100 to 145 with a mortality risk of 5%10%; and >145 with a mortality risk of 10% or more. See text and Escobar et al4 for more details. ∥Numbers for patients who survived last hospitalization to discharge are available upon request.
N	89,269	5963	2880	4310
Age (mean SD)	61.26 18.62	62.25 18.13	66.12 16.20	64.45 15.91
Male (n, %)	37,228 (41.70%)	3091 (51.84%)	1416 (49.17%)	2378 (55.17%)
LAPS (mean SD)	13.02 15.79	32.72 24.85	24.83 21.53	11.79 18.16
COPS(mean SD)	67.25 51.42	73.88 57.42	86.33 59.33	78.44 52.49
% Predicted mortality risk (mean SD)	1.93% 3.98%	7.69% 12.59%	5.23% 7.70%	3.66% 6.81%
Survived first hospitalization to discharge^∥	88,479 (99.12%)	5336 (89.49%)	2316 (80.42%)	4063 (94.27%)
Care order on admission
Full code	78,877 (88.36%)	5198 (87.17%)	2598 (90.21%)	4097 (95.06%)
Partial code	664 (0.74%)	156 (2.62%)	50 (1.74%)	27 (0.63%)
Comfort care	21 (0.02%)	2 (0.03%)	0 (0%)	0 (0%)
DNR	8227 (9.22%)	539 (9.04%)	219 (7.60%)	161 (3.74%)
Comfort care and DNR	229 (0.26%)	9 (0.15%)	2 (0.07%)	2 (0.05%)
No order	1251 (1.40%)	59 (0.99%)	11 (0.38%)	23 (0.53%)
Admission diagnosis (n, %)
Pneumonia	2385 (2.67%)	258 (4.33%)	242 (8.40%)	68 (1.58%)
Sepsis	5822 (6.52%)	503 (8.44%)	279 (9.69%)	169 (3.92%)
GI bleeding	9938 (11.13%)	616 (10.33%)	333 (11.56%)	290 (6.73%)
Cancer	2845 (3.19%)	14 (0.23%)	95 (3.30%)	492 (11.42%)
Total hospital length of stay (days SD)	3.08 3.29	5.37 7.50	12.16 13.12	8.06 9.53

Table 2.Event and Comparison Shifts
Predictor	Event Shifts	Comparison Shifts	P
NOTE: Code status, vital sign, and laboratory values measures closest to the start of the shift (7 _AM or 7 _PM) are used. Abbreviations: COPS, COmorbidity Point Score; ICU, intensive care unit; LAPS, Laboratory Acute Physiology Score; MEWS(re), Modified Early Warning Score (retrospective electronic); SD, standard deviation. * LAPS; see Table 1, text, and Escobar et al4 for more details. COPS; see Table 1, text, and Escobar et al4 for more details. Refers to patients who had an active full code order at the start of the sampling time frame. See text for explanation of sampling time frame, and how both cases and controls could have been in the ICU. ∥ See text for explanation of how both cases and controls could have experienced an unplanned transfer to the ICU. MEWS(re); see text and Subbe et al10 for a description of this score. # (Anion gap bicarbonate) 100. ** Rates of missing data for vital signs are not shown because <3% of the shifts were missing these data.
Number	4036	39,782
Age (mean SD)	67.19 15.25	65.41 17.40	<0.001
Male (n, %)	2007 (49.73%)	17,709 (44.52%)	<0.001
Day shift	1364 (33.80%)	17,714 (44.53%)	<0.001
LAPS*	27.89 22.10	20.49 20.16	<0.001
COPS	116.33 72.31	100.81 68.44	<0.001
Full code (n, %)	3496 (86.2%)	32,156 (80.8%)	<0.001
ICU shift during hospitalization	3964 (98.22%)	7197 (18.09%)	<0.001
Unplanned transfer to ICU during hospitalization∥	353 (8.8%)	1466 (3.7%)	<0.001
Temperature (mean SD)	98.15 (1.13)	98.10 (0.85)	0.009
Heart rate (mean SD)	90.30 (20.48)	79.86 (5.27)	<0.001
Respiratory rate (mean SD)	20.36 (3.70)	18.87 (1.79)	<0.001
Systolic blood pressure (mean SD)	123.65 (23.26)	126.21 (19.88)	<0.001
Diastolic blood pressure (mean SD)	68.38 (14.49)	69.46 (11.95)	<0.001
Oxygen saturation (mean SD)	95.72% (3.00)	96.47 % (2.26)	<0.001
MEWS(re) (mean SD)	3.64 (2.02)	2.34 (1.61)	<0.001
% <5	74.86%	92.79%
% 5	25.14%	7.21%	<0.001
Proxy for measured lactate# (mean SD)	36.85 (28.24)	28.73 (16.74)	<0.001
% Missing in 24 hr before start of shift**	17.91%	28.78%	<0.001
Blood urea nitrogen (mean SD)	32.03 (25.39)	22.72 (18.9)	<0.001
% Missing in 24 hr before start of shift	19.67%	20.90%	<0.001
White blood cell count 1000 (mean SD)	12.33 (11.42)	9.83 (6.58)	<0.001
% Missing in 24 hr before start of shift	21.43%	30.98%	<0.001
Hematocrit (mean SD)	33.08 (6.28)	33.07 (5.25)	0.978
% Missing in 24 hr before start of shift	19.87%	29.55%	<0.001

Table 3.Variables Included in Final Electronic Medical Record‐Based Models
Variable	Description
Abbreviations: COPS, COmorbidity Point Score; LAPS, Laboratory Acute Physiology Score; LOS, length of stay. * LAPS based on 14 laboratory test results obtained in the 72 hr preceding hospitalization. See text and Escobar et al4 for details. COPS based on a patient's diagnoses in the 12 mo preceding hospitalization. See text and Escobar et al4 for details. Indicator variable (for patients in whom a COPS could not be obtained) also included in models. See text and Supporting Information, Appendices, in the online version of this article for details on imputation strategy employed when values were missing. See Wrenn14 and Rocktaeschel et al16 for justification for use of the combination of anion gap and serum bicarbonate.
Directive status	Full code or not full code
LAPS*	Admission physiologic severity of illness score (continuous variable ranging from 0 to 256). Standardized and included as LAPS and LAPS squared
COPS	Comorbidity burden score (continuous variable ranging from 0 to 701). Standardized and included as COPS and COPS squared.
COPS status	Indicator for absent comorbidity data
LOS at T₀	Length of stay in the hospital (total time in hours) at the T₀; standardized.
T₀ time of day	7 _AM or 7 _PM
Temperature	Worst (highest) temperature in 24 hr preceding T₀; variability in temperature in 24 hr preceding T₀.
Heart rate	Most recent heart rate in 24 hr preceding T₀; variability in heart rate in 24 hr preceding T₀.
Respiratory rate	Most recent respiratory rate in 24 hr preceding T₀; worst (highest) respiratory rate in 24 hr preceding T₀; variability in respiratory rate in 24 hr preceding T₀.
Diastolic blood pressure	Most recent diastolic blood pressure in 24 hr preceding T₀ transformed by subtracting 70 from the actual value and squaring the result. Any value above 2000 is subsequently then set to 2000, yielding a continuous variable ranging from 0 to 2000.
Systolic pressure	Variability in systolic blood pressure in 24 hr preceding T₀.

Pulse oximetry	Worst (lowest) oxygen saturation in 24 hr preceding T₀; variability in oxygen saturation in 24 hr preceding T₀.
Neurological status	Most recent neurological status check in 24 hr preceding T₀.
Laboratory tests	Blood urea nitrogen
	Proxy for measured lactate = (anion gap serum bicarbonate) 100
	Hematocrit
	Total white blood cell count

Table 4.Best and Worst Performing Submodels in the Validation Dataset
	No. of Shifts in Validation Dataset		c‐Statistic
Diagnoses Group*	Event	Comparison	MEWS(re)	EMR Model
Abbreviations: EMR, electronic medical record; GI, gastrointestinal; MEWS(re), Modified Early Warning Score (retrospective electronic). * Specific International Classification of Diseases (ICD) codes used are detailed in the Supporting Information, Appendices, in the online version of this article. MEWS(re); see text, Supporting Information, Appendices, in the online version of this article, and Subbe et al10 for more details. Model based on comprehensive data from EMR; see text, Table 3, and Supporting Information, Appendices, in the online version of this article for more details. This group of diagnoses includes appendicitis, cholecystitis, cholangitis, hernias, and pancreatic disorders. ∥ This group of diagnoses includes: gastrointestinal hemorrhage, miscellaneous disorders affecting the stomach and duodenum, diverticulitis, abdominal symptoms, nausea with vomiting, and blood in stool. This group of diagnoses includes inflammatory bowel disease, malabsorption syndromes, gastrointestinal obstruction, and enteritides.
Acute myocardial infarction	36	169	0.541	0.572
Diseases of pulmonary circulation and cardiac dysrhythmias	40	329	0.565	0.645
Seizure disorders	45	497	0.594	0.647
Rule out myocardial infarction	77	727	0.602	0.648
Pneumonia	163	847	0.741	0.801
GI diagnoses, set A	58	942	0.755	0.803
GI diagnoses, set B∥	256	2,610	0.772	0.806
GI diagnoses, set C	46	520	0.792	0.841
All diagnosis	2,032	20,106	0.698	0.775

DISCUSSION

Acknowledgements

References

Barnett MJ,Kaboli PJ,Sirio CA,Rosenthal GE.Day of the week of intensive care admission and patient outcomes: a multisite regional evaluation.Med Care.2002;40(6):530–539.
Ensminger SA,Morales IJ,Peters SG, et al.The hospital mortality of patients admitted to the ICU on weekends.Chest.2004;126(4):1292–1298.
Luyt CE,Combes A,Aegerter P, et al.Mortality among patients admitted to intensive care units during weekday day shifts compared with “off” hours.Crit Care Med.2007;35(1):3–11.
Escobar G,Greene J,Scheirer P,Gardner M,Draper D,Kipnis P.Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases.Med Care.2008;46(3):232–239.
van Walraven C,Escobar GJ,Greene JD,Forster AJ.The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population.J Clin Epidemiol.2010;63(7):798–803.
Escobar GJ,Greene JD,Gardner MN,Marelich GP,Quick B,Kipnis P.Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS).J Hosp Med.2011;6(2):74–80.
Chambrin MC,Ravaux P,Calvelo‐Aros D,Jaborska A,Chopin C,Boniface B.Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis.Intensive Care Med.1999;25(12):1360–1366.
Saria S,Rajani AK,Gould J,Koller D,Penn AA.Integration of early physiological responses predicts later illness severity in preterm infants.Sci Transl Med.2010;2(48):48ra65.
Subbe CP,Gao H,Harrison DA.Reproducibility of physiological track‐and‐trigger warning systems for identifying at‐risk patients on the ward.Intensive Care Med.2007;33(4):619–624.
Subbe CP,Kruger M,Rutherford P,Gemmel L.Validation of a Modified Early Warning Score in medical admissions.Q J Med.2001;94:521–526.
Subbe CP,Davies RG,Williams E,Rutherford P,Gemmell L.Effect of introducing the Modified Early Warning score on clinical outcomes, cardio‐pulmonary arrests and intensive care utilisation in acute medical admissions.Anaesthesia.2003;58(8):797–802.
MERIT Study Investigators.Introduction of the medical emergency team (MET) system: a cluster‐randomized controlled trial.Lancet.2005;365(9477):2091–2097.
Keller AS,Kirkland LL,Rajasekaran SY,Cha S,Rady MY,Huddleston JM.Unplanned transfers to the intensive care unit: the role of the shock index.J Hosp Med.2010;5(8):460–465.
Wrenn K.The delta (delta) gap: an approach to mixed acid‐base disorders.Ann Emerg Med.1990;19(11):1310–1313.
Williamson JC.Acid‐base disorders: classification and management strategies.Am Fam Physician.1995;52(2):584–590.
Rocktaeschel J,Morimatsu H,Uchino S,Bellomo R.Unmeasured anions in critically ill patients: can they predict mortality?Crit Care Med.2003;31(8):2131–2136.
Cook NR.Use and misuse of the receiver operating characteristic curve in risk prediction.Circulation.2007;115(7):928–935.
Bapoje SR,Gaudiani JL,Narayanan V,Albert RK.Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care.J Hosp Med.2011;6(2):68–72.
Litvak E,Pronovost PJ.Rethinking rapid response teams.JAMA.2010;304(12):1375–1376.
Winters BD,Pham J,Pronovost PJ.Rapid response teams—walk, don't run.JAMA.2006;296(13):1645–1647.
Charlson ME,Pompei P,Ales KL,MacKenzie CR.A new method of classifying prognostic comorbidity in longitudinal populations: development and validation.J Chronic Dis.1987;40:373–383.
Institute for Healthcare Improvement.Early Warning Systems:The Next Level of Rapid Response.2011. http://www.ihi.org/IHI/Programs/AudioAndWebPrograms/ExpeditionEarlyWarningSystemsTheNextLevelofRapidResponse.htm?player=wmp. Accessed 4/6/11.
Bowes WA.Assessing readiness for meeting meaningful use: identifying electronic health record functionality and measuring levels of adoption.AMIA Annu Symp Proc.2010;2010:66–70.
Medicare and Medicaid Programs;Electronic Health Record Incentive Program. Final Rule.Fed Reg.2010;75(144):44313–44588.

References

Barnett MJ,Kaboli PJ,Sirio CA,Rosenthal GE.Day of the week of intensive care admission and patient outcomes: a multisite regional evaluation.Med Care.2002;40(6):530–539.
Ensminger SA,Morales IJ,Peters SG, et al.The hospital mortality of patients admitted to the ICU on weekends.Chest.2004;126(4):1292–1298.
Luyt CE,Combes A,Aegerter P, et al.Mortality among patients admitted to intensive care units during weekday day shifts compared with “off” hours.Crit Care Med.2007;35(1):3–11.
Escobar G,Greene J,Scheirer P,Gardner M,Draper D,Kipnis P.Risk adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases.Med Care.2008;46(3):232–239.
van Walraven C,Escobar GJ,Greene JD,Forster AJ.The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population.J Clin Epidemiol.2010;63(7):798–803.
Escobar GJ,Greene JD,Gardner MN,Marelich GP,Quick B,Kipnis P.Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS).J Hosp Med.2011;6(2):74–80.
Chambrin MC,Ravaux P,Calvelo‐Aros D,Jaborska A,Chopin C,Boniface B.Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis.Intensive Care Med.1999;25(12):1360–1366.
Saria S,Rajani AK,Gould J,Koller D,Penn AA.Integration of early physiological responses predicts later illness severity in preterm infants.Sci Transl Med.2010;2(48):48ra65.
Subbe CP,Gao H,Harrison DA.Reproducibility of physiological track‐and‐trigger warning systems for identifying at‐risk patients on the ward.Intensive Care Med.2007;33(4):619–624.
Subbe CP,Kruger M,Rutherford P,Gemmel L.Validation of a Modified Early Warning Score in medical admissions.Q J Med.2001;94:521–526.
Subbe CP,Davies RG,Williams E,Rutherford P,Gemmell L.Effect of introducing the Modified Early Warning score on clinical outcomes, cardio‐pulmonary arrests and intensive care utilisation in acute medical admissions.Anaesthesia.2003;58(8):797–802.
MERIT Study Investigators.Introduction of the medical emergency team (MET) system: a cluster‐randomized controlled trial.Lancet.2005;365(9477):2091–2097.
Keller AS,Kirkland LL,Rajasekaran SY,Cha S,Rady MY,Huddleston JM.Unplanned transfers to the intensive care unit: the role of the shock index.J Hosp Med.2010;5(8):460–465.
Wrenn K.The delta (delta) gap: an approach to mixed acid‐base disorders.Ann Emerg Med.1990;19(11):1310–1313.
Williamson JC.Acid‐base disorders: classification and management strategies.Am Fam Physician.1995;52(2):584–590.
Rocktaeschel J,Morimatsu H,Uchino S,Bellomo R.Unmeasured anions in critically ill patients: can they predict mortality?Crit Care Med.2003;31(8):2131–2136.
Cook NR.Use and misuse of the receiver operating characteristic curve in risk prediction.Circulation.2007;115(7):928–935.
Bapoje SR,Gaudiani JL,Narayanan V,Albert RK.Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care.J Hosp Med.2011;6(2):68–72.
Litvak E,Pronovost PJ.Rethinking rapid response teams.JAMA.2010;304(12):1375–1376.
Winters BD,Pham J,Pronovost PJ.Rapid response teams—walk, don't run.JAMA.2006;296(13):1645–1647.
Charlson ME,Pompei P,Ales KL,MacKenzie CR.A new method of classifying prognostic comorbidity in longitudinal populations: development and validation.J Chronic Dis.1987;40:373–383.
Institute for Healthcare Improvement.Early Warning Systems:The Next Level of Rapid Response.2011. http://www.ihi.org/IHI/Programs/AudioAndWebPrograms/ExpeditionEarlyWarningSystemsTheNextLevelofRapidResponse.htm?player=wmp. Accessed 4/6/11.
Bowes WA.Assessing readiness for meeting meaningful use: identifying electronic health record functionality and measuring levels of adoption.AMIA Annu Symp Proc.2010;2010:66–70.
Medicare and Medicaid Programs;Electronic Health Record Incentive Program. Final Rule.Fed Reg.2010;75(144):44313–44588.

Issue

Journal of Hospital Medicine - 7(5)

Issue

Journal of Hospital Medicine - 7(5)

Page Number

388-395

Page Number

388-395

Publications

Publications

Article Type

Display Headline

Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record

Display Headline

Early detection of impending physiologic deterioration among patients who are not in intensive care: Development of predictive models using data from an automated electronic medical record

Sections

Article Source

Disallow All Ads

Corresponding Author

Correspondence Location

Division of Research, Kaiser Permanente Medical Care Program, 2000 Broadway Ave, Oakland, CA 94612===

Content Gating

No Gating (article Unlocked/Free)

Alternative CME

Teaser Media

Article PDF Media

Media Files