User login
Severe‐Sepsis Screening Tool
Sepsis remains a significant healthcare burden and is the sixth most common reason for hospitalization in the United States. For patients presenting with severe sepsis, mortality rates are approximately 30%,[1, 2] and sepsis remains the most expensive reason for hospitalization. In 2009, septicemia accounted for nearly $15.4 billion in aggregate hospital costs.[2]
Early identification of sepsis and the timely implementation of goal‐directed therapy significantly decrease sepsis‐related mortality and are cost‐effective,[3, 4, 5] highlighting the need for new clinical strategies to aid in early diagnosis. To date, most studies have focused on the screening and management of sepsis in the emergency department and intensive care unit (ICU),[6, 7] and less is known about the benefits of screening in non‐ICU settings. In the non‐ICU setting, conditions may go unrecognized and treatments delayed. Evidence suggests that patients diagnosed with severe sepsis in the non‐ICU setting are almost twice as likely to die as those diagnosed in an emergency department.[8, 9]
Application of a sepsis screening tool to both medical and surgical patients poses an additional challenge that may impact the screen's performance. The specificity may be compromised by noninfectious causes of systemic inflammatory response syndrome (SIRS) commonly seen in the postsurgical patient. For example, the tachycardia and fever often seen in the postoperative patient are sufficient to qualify for SIRS, making the diagnosis of sepsis more challenging. The purpose of this study was to examine the performance of a nurse‐driven, simple sepsis screening tool in a mixed medical and surgical non‐ICU setting.
METHODS
Setting
This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.
Nurse Education
Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.
In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.
Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.
The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.
Screening Tool
A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.
The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO2) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.
If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.
If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.
Data Collection
Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.
Performance Measurement
The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.
Clinical Activities
To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.
Statistical Analysis
To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.
RESULTS
Over a 1‐month time period, 2143 screens were completed on 245 patients (169 surgical, 76 medical). The overall incidence of sepsis on the treatment unit during this time period was 9%. Surgical patients had an 8.9% incidence of sepsis, and medical patients had an incidence of 9.2%.
Screening tool performance is presented in Table 1. The screening tool had 95.5% sensitivity and 91.9% specificity, with no significant differences in performance between surgical and medical patients. The overall negative predictive value was 99.5%, also with comparable performance in both surgical and medical patients (P = 0.89). The overall positive predictive value (PPV) was 70% in medical patients and 48% in surgical patients (P = 0.12). Screening tool accuracy for medical and surgical patients was 92%.
| Overall, N = 245 (95% CI) | Surgery, N = 169 (95% CI) | Medicine, N = 76 (95% CI) | P Value* | |
|---|---|---|---|---|
| ||||
| Sensitivity | 95.5% (75%‐99.7%) | 93% (66%‐99.6%) | 100% (56%‐100%) | 0.17 |
| Specificity | 91.9% (87%‐95%) | 90% (84%‐94%) | 95% (87%‐99%) | 0.48 |
| NPV | 99.5% (81%‐100%) | 99.3% (71%‐100%) | 100% (67%‐100%) | 0.89 |
| PPV | 53.8% (39%‐70%) | 48% (23%‐73%) | 70% (30%‐100%) | 0.12 |
| LR+ | 11.8 | 9.3 | 20 | |
| LR | 0.05 | 0.08 | 0 | |
| Confirmed patient diagnosis, overall | ||||
| Sepsis | No sepsis | |||
| Screen positive | 21 (TP) | 18 (FP) | ||
| Screen negative | 1 (FN) | 205 (TN) | ||
| Confirmed patient diagnosis, medicine | ||||
| Sepsis | No sepsis | |||
| Screen positive | 7 (TP) | 3 (FP) | ||
| Screen negative | 0 (FN) | 66 (TN) | ||
| Confirmed patient diagnosis, surgery | ||||
| Sepsis | No sepsis | |||
| Screen positive | 14 (TP) | 15 (FP) | ||
| Screen negative | 1 (FN) | 139 (TN) | ||
Clinical Activities
Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.
Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.
| Patient Characteristics | Surgery (Positive) | Medicine (Positive) | Surgery (Negative) | Medicine (Negative) | P Value |
|---|---|---|---|---|---|
| |||||
| No. | 26 | 7 | 20 | 10 | |
| Age, y, mean | 57.8 ( 16.5) | 72.4 ( 16.8) | 64.6 ( 19.4) | 63.6 ( 16.8) | 0.25 |
| % Male (no.) | 50% (13) | 57% (4) | 60% (12) | 60% (6) | 0.27 |
| Length of stay, d, median (IQR) | 9 (716.7) | 7 (5.511.5) | 11 (7.722) | 8 (421) | 0.38 |
| No. of PODs until first positive screen, d, median (IQR) | 2 (13) | N/A | N/A | N/A | |
| % Mortality (no.) | 0% | 14% (1) | 5% (1) | 10% (1) | 0.19 |
Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.
We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).
| Intervention and Group | Proportion | P Value |
|---|---|---|
| ||
| Before screening test | ||
| Antibiotics | 0.066 | |
| Positive screen | 45% | |
| Negative screen | 23% | |
| Lactate | 0.837 | |
| Positive screen | 15% | |
| Negative screen | 13% | |
| Blood culture | 0.181 | |
| Positive screen | 18% | |
| Negative screen | 17% | |
| Fluid administration | 0.564 | |
| Positive screen | 6% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.337 | |
| Positive screen | 3% | |
| Negative screen | 0% | |
| After screening test | ||
| Antibiotics | 0.006 | |
| Positive screen | 58% | |
| Negative screen | 23% | |
| Lactate | 0.018 | |
| Positive screen | 36% | |
| Negative screen | 13% | |
| Blood Culture | 0.002 | |
| Positive screen | 24% | |
| Negative screen | 17% | |
| Fluid administration | 0.112 | |
| Positive screen | 24% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.175 | |
| Positive screen | 9% | |
| Negative screen | 3% | |
DISCUSSION
Improving recognition and time to treatment of sepsis in a non‐ICU setting is an important step toward decreasing sepsis‐related mortality. Lundberg and colleagues found that mortality rates for patients diagnosed with septic shock on a general ward were higher than for patients diagnosed in the ICU, even though ward patients were younger and healthier at baseline.[8] For ward patients, treatment delays were most profound in initiating vasoactive therapies, and minor delays were encountered in initiating fluid resuscitation. In their international study on the impact of early goal‐directed therapy guidelines, Levy and colleagues found that patients diagnosed with severe sepsis on the wards were almost twice as likely to die as patients diagnosed with sepsis in the emergency department.[9]
We are the first to report about an accurate nurse‐driven SIRS‐based sepsis screening protocol that is effective in the early identification of sepsis in both medical and surgical patients in an intermediate care setting. We found no significant difference in the screening tool performance between the medical and surgical cohorts. This is an important comparison given that SIRS criteria alone can be nonspecific in the postoperative population, where it is common to have hemodynamic changes, elevation of inflammatory markers, and fevers from noninfectious sources.
Our sepsis screening tool was designed in 3 tiers to improve its specificity. The first tier was based strictly on SIRS criteria (eg, tachycardia or fever), whereas the second and third tiers served to increase the specificity of the screening tool by instructing the evaluator to assess possible sources of infection and assess for objective signs of organ dysfunction. We relied heavily on the nursing staff to assess for the presence or absence of infection and believe that the educational component prior to initiating the screening protocol was vital.
EMR‐based screening tools that rely purely on physiologic data have been considered for the early detection and management of sepsis, although they lack the specificity gained through the incorporation of clinical judgment.[13] Sawyer and colleagues report using a real‐time EMR‐based method for early sepsis detection in non‐ICU patients that is based solely on objective measures; however, their PPV was only 19.5%. The model we describe in this study is one that incorporates real‐time physiologic data available from an EMR coupled with the clinical judgment of a bedside registered nurse. As our data suggest, this provides a screen that is both sensitive and specific.
It is interesting to note that in our assessment of clinical action taken 8 hours after a positive screening test (the interval after which a new screening test was performed), the rate of diagnostic workup and/or treatment for sepsis was relatively low. One reason for this could have been that the treating team had suspicion for sepsis prior to a positive screen and had already initiated clinical action. Of the 51 recorded clinical actions taken around the time of a positive screen, the majority (67%) occurred before the screening result. It is also possible that clinical action was not pursued because the treatment team disagreed with a diagnosis of sepsis. Of all the false positive screening cases, manual chart review confirmed that these patients did not have sepsis, nor did they develop sepsis during their index hospital stay. Last, we only recorded clinical actions taken within 8 hours of the first positive screen for sepsis and measured 5 very specific actions. Thus, our analysis may have missed actions taken after 8 hours or actions that differed from the 5 we chose to assess.
Even with the apparently low levels of new clinical activity after a positive screen, when compared to patients who screened negative for sepsis, a significantly higher number of patients who had a positive screen received antibiotics, had lactate measured, and had blood cultures drawn. We did not find a significant difference in the proportion of patients receiving a sepsis‐related clinical action before a screening result (positive or negative), which suggests that a positive screening test may have led to increased clinical action.
A limitation of our study is its small size and that it was conducted in 1 pilot unit. Additionally, our retrospective analysis of clinical care inhibited our ability to fully understand a patient's clinical course or retrieve missing data points. A related limitation is that we could not ascertain how often the screening tool did not identify a case of sepsis before it was clinically diagnosed. Assessing the temporal performance of our screening tool is of great interest and may be more easily performed using an electronic version of the screening tool, which is currently in development.
Using ICD‐9 codes to determine the true‐negative cohort is another limitation of our study. It is well documented that use of administrative data can lead to inaccurate classification of patients.[14] To address this, we performed random audits of 30 test‐negative patients. In doing so we did not find any errors in classification.
Although we did not find a significant difference in screening tool performance between surgical and medical patients, the PPV of the tool was lower in the surgical population (48%) compared to the medical population (70%). The lower PPV observed in surgical patients could be attributable to an overall lower incidence of sepsis in this cohort as well as possible errors in initial assessment of infection, which can be difficult in postsurgical patients. Our retrospective analysis included data from the early months of the screening protocol, a time in which nursing staff was still developing clinical acumen in identifying sepsis. However, this could have led nurses to either overestimate or underestimate the presence of infection in either patient group.
Suspicion for infection is the cornerstone definition of sepsis, and in our screening protocol nurses were charged with making this decision based on their knowledge of the patient's clinical course and current status. Issues concerning nurses' recognition of infection symptoms are an area of opportunity for further research and education and could aid in improving PPV. Clinical judgment could be further bolstered by adding promising laboratory tests such as C‐reactive protein or procalcitonin as objective adjuncts to an initial assessment for sepsis,[15] which could potentially increase screening test PPV.
CONCLUSIONS
A simple screening tool for sepsis performed by the bedside nurse can provide a means to successfully identify sepsis early and lead to more timely diagnostics and treatment in both medical and surgical patients in an intermediate care setting.
ACKNOWLEDGEMENTS
The authors thank Eileen Pummer, quality manager for the sepsis team; Pauline Regner, patient care manager of the pilot study unit; and the nurses who contributed to the screening tool design team and data collection. The authors acknowledge Pooja Loftus for her statistical expertise, and Isabella Chu for her review of the manuscript. Disclosures: Presented as a poster at the 31st Annual Meeting of the Surgical Infection Society, Palm Beach, Florida, May 2011. The authors report no conflicts of interest.
- , , , , , Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
- , , Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. Agency for Healthcare Research and Quality. Available at: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb122.pdf. Published October 2011. Accessed on September 4, 2012.
- , , , Economic implications of an evidence‐based sepsis protocol: can we improve outcomes and lower costs? Crit Care Med. 2007;35(5):1257–1262.
- , , , , , Late compliance with the sepsis resuscitation bundle: impact on mortality. Shock. 2011;36(6):542–547.
- , , , , , The costs and cost‐effectiveness of an integrated sepsis treatment protocol. Crit Care Med. 2008;36(4):1168–1174.
- , , A simple prediction algorithm for bacteraemia in patients with acute febrile illness. QJM. 2005;98(11):813–820.
- , , , et al. Validation of a screening tool for the early identification of sepsis. J Trauma. 2009;66(6):1539–1546; discussion 1546–1547.
- , , , et al. Septic shock: an analysis of outcomes for patients with onset on hospital wards versus intensive care units. Crit Care Med. 1998;26(6):1020–1024.
- , , , et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
- Institute of Healthcare Improvement. Evaluation for severe sepsis screening tool. Surviving Sepsis Campaign. Available at: http://www.survivingsepsis.org/About_the_Campaign/Documents/evaluationforseveresepsisscreeningtool.pdf. Accessed on September 30, 2012.
- , , , et al. Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: results of a three‐year follow‐up quasi‐experimental study. Crit Care Med. 2010;38(4):1036–1043.
- , , , et al. Reduction of the severe sepsis or septic shock associated mortality by reinforcement of the recommendations bundle: a multicenter study. Ann Fr Anesth Reanim. 2010;29(9):621–628.
- , , , et al. Implementation of a real‐time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011;39(3):469–473.
- , , , Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual. 2005;20(6):319–328.
- , , , , , Comparison of procalcitonin and C‐reactive protein as markers of sepsis. Crit Care Med. 2003;31(6):1737–1741.
Sepsis remains a significant healthcare burden and is the sixth most common reason for hospitalization in the United States. For patients presenting with severe sepsis, mortality rates are approximately 30%,[1, 2] and sepsis remains the most expensive reason for hospitalization. In 2009, septicemia accounted for nearly $15.4 billion in aggregate hospital costs.[2]
Early identification of sepsis and the timely implementation of goal‐directed therapy significantly decrease sepsis‐related mortality and are cost‐effective,[3, 4, 5] highlighting the need for new clinical strategies to aid in early diagnosis. To date, most studies have focused on the screening and management of sepsis in the emergency department and intensive care unit (ICU),[6, 7] and less is known about the benefits of screening in non‐ICU settings. In the non‐ICU setting, conditions may go unrecognized and treatments delayed. Evidence suggests that patients diagnosed with severe sepsis in the non‐ICU setting are almost twice as likely to die as those diagnosed in an emergency department.[8, 9]
Application of a sepsis screening tool to both medical and surgical patients poses an additional challenge that may impact the screen's performance. The specificity may be compromised by noninfectious causes of systemic inflammatory response syndrome (SIRS) commonly seen in the postsurgical patient. For example, the tachycardia and fever often seen in the postoperative patient are sufficient to qualify for SIRS, making the diagnosis of sepsis more challenging. The purpose of this study was to examine the performance of a nurse‐driven, simple sepsis screening tool in a mixed medical and surgical non‐ICU setting.
METHODS
Setting
This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.
Nurse Education
Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.
In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.
Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.
The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.
Screening Tool
A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.
The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO2) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.
If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.
If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.
Data Collection
Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.
Performance Measurement
The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.
Clinical Activities
To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.
Statistical Analysis
To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.
RESULTS
Over a 1‐month time period, 2143 screens were completed on 245 patients (169 surgical, 76 medical). The overall incidence of sepsis on the treatment unit during this time period was 9%. Surgical patients had an 8.9% incidence of sepsis, and medical patients had an incidence of 9.2%.
Screening tool performance is presented in Table 1. The screening tool had 95.5% sensitivity and 91.9% specificity, with no significant differences in performance between surgical and medical patients. The overall negative predictive value was 99.5%, also with comparable performance in both surgical and medical patients (P = 0.89). The overall positive predictive value (PPV) was 70% in medical patients and 48% in surgical patients (P = 0.12). Screening tool accuracy for medical and surgical patients was 92%.
| Overall, N = 245 (95% CI) | Surgery, N = 169 (95% CI) | Medicine, N = 76 (95% CI) | P Value* | |
|---|---|---|---|---|
| ||||
| Sensitivity | 95.5% (75%‐99.7%) | 93% (66%‐99.6%) | 100% (56%‐100%) | 0.17 |
| Specificity | 91.9% (87%‐95%) | 90% (84%‐94%) | 95% (87%‐99%) | 0.48 |
| NPV | 99.5% (81%‐100%) | 99.3% (71%‐100%) | 100% (67%‐100%) | 0.89 |
| PPV | 53.8% (39%‐70%) | 48% (23%‐73%) | 70% (30%‐100%) | 0.12 |
| LR+ | 11.8 | 9.3 | 20 | |
| LR | 0.05 | 0.08 | 0 | |
| Confirmed patient diagnosis, overall | ||||
| Sepsis | No sepsis | |||
| Screen positive | 21 (TP) | 18 (FP) | ||
| Screen negative | 1 (FN) | 205 (TN) | ||
| Confirmed patient diagnosis, medicine | ||||
| Sepsis | No sepsis | |||
| Screen positive | 7 (TP) | 3 (FP) | ||
| Screen negative | 0 (FN) | 66 (TN) | ||
| Confirmed patient diagnosis, surgery | ||||
| Sepsis | No sepsis | |||
| Screen positive | 14 (TP) | 15 (FP) | ||
| Screen negative | 1 (FN) | 139 (TN) | ||
Clinical Activities
Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.
Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.
| Patient Characteristics | Surgery (Positive) | Medicine (Positive) | Surgery (Negative) | Medicine (Negative) | P Value |
|---|---|---|---|---|---|
| |||||
| No. | 26 | 7 | 20 | 10 | |
| Age, y, mean | 57.8 ( 16.5) | 72.4 ( 16.8) | 64.6 ( 19.4) | 63.6 ( 16.8) | 0.25 |
| % Male (no.) | 50% (13) | 57% (4) | 60% (12) | 60% (6) | 0.27 |
| Length of stay, d, median (IQR) | 9 (716.7) | 7 (5.511.5) | 11 (7.722) | 8 (421) | 0.38 |
| No. of PODs until first positive screen, d, median (IQR) | 2 (13) | N/A | N/A | N/A | |
| % Mortality (no.) | 0% | 14% (1) | 5% (1) | 10% (1) | 0.19 |
Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.
We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).
| Intervention and Group | Proportion | P Value |
|---|---|---|
| ||
| Before screening test | ||
| Antibiotics | 0.066 | |
| Positive screen | 45% | |
| Negative screen | 23% | |
| Lactate | 0.837 | |
| Positive screen | 15% | |
| Negative screen | 13% | |
| Blood culture | 0.181 | |
| Positive screen | 18% | |
| Negative screen | 17% | |
| Fluid administration | 0.564 | |
| Positive screen | 6% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.337 | |
| Positive screen | 3% | |
| Negative screen | 0% | |
| After screening test | ||
| Antibiotics | 0.006 | |
| Positive screen | 58% | |
| Negative screen | 23% | |
| Lactate | 0.018 | |
| Positive screen | 36% | |
| Negative screen | 13% | |
| Blood Culture | 0.002 | |
| Positive screen | 24% | |
| Negative screen | 17% | |
| Fluid administration | 0.112 | |
| Positive screen | 24% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.175 | |
| Positive screen | 9% | |
| Negative screen | 3% | |
DISCUSSION
Improving recognition and time to treatment of sepsis in a non‐ICU setting is an important step toward decreasing sepsis‐related mortality. Lundberg and colleagues found that mortality rates for patients diagnosed with septic shock on a general ward were higher than for patients diagnosed in the ICU, even though ward patients were younger and healthier at baseline.[8] For ward patients, treatment delays were most profound in initiating vasoactive therapies, and minor delays were encountered in initiating fluid resuscitation. In their international study on the impact of early goal‐directed therapy guidelines, Levy and colleagues found that patients diagnosed with severe sepsis on the wards were almost twice as likely to die as patients diagnosed with sepsis in the emergency department.[9]
We are the first to report about an accurate nurse‐driven SIRS‐based sepsis screening protocol that is effective in the early identification of sepsis in both medical and surgical patients in an intermediate care setting. We found no significant difference in the screening tool performance between the medical and surgical cohorts. This is an important comparison given that SIRS criteria alone can be nonspecific in the postoperative population, where it is common to have hemodynamic changes, elevation of inflammatory markers, and fevers from noninfectious sources.
Our sepsis screening tool was designed in 3 tiers to improve its specificity. The first tier was based strictly on SIRS criteria (eg, tachycardia or fever), whereas the second and third tiers served to increase the specificity of the screening tool by instructing the evaluator to assess possible sources of infection and assess for objective signs of organ dysfunction. We relied heavily on the nursing staff to assess for the presence or absence of infection and believe that the educational component prior to initiating the screening protocol was vital.
EMR‐based screening tools that rely purely on physiologic data have been considered for the early detection and management of sepsis, although they lack the specificity gained through the incorporation of clinical judgment.[13] Sawyer and colleagues report using a real‐time EMR‐based method for early sepsis detection in non‐ICU patients that is based solely on objective measures; however, their PPV was only 19.5%. The model we describe in this study is one that incorporates real‐time physiologic data available from an EMR coupled with the clinical judgment of a bedside registered nurse. As our data suggest, this provides a screen that is both sensitive and specific.
It is interesting to note that in our assessment of clinical action taken 8 hours after a positive screening test (the interval after which a new screening test was performed), the rate of diagnostic workup and/or treatment for sepsis was relatively low. One reason for this could have been that the treating team had suspicion for sepsis prior to a positive screen and had already initiated clinical action. Of the 51 recorded clinical actions taken around the time of a positive screen, the majority (67%) occurred before the screening result. It is also possible that clinical action was not pursued because the treatment team disagreed with a diagnosis of sepsis. Of all the false positive screening cases, manual chart review confirmed that these patients did not have sepsis, nor did they develop sepsis during their index hospital stay. Last, we only recorded clinical actions taken within 8 hours of the first positive screen for sepsis and measured 5 very specific actions. Thus, our analysis may have missed actions taken after 8 hours or actions that differed from the 5 we chose to assess.
Even with the apparently low levels of new clinical activity after a positive screen, when compared to patients who screened negative for sepsis, a significantly higher number of patients who had a positive screen received antibiotics, had lactate measured, and had blood cultures drawn. We did not find a significant difference in the proportion of patients receiving a sepsis‐related clinical action before a screening result (positive or negative), which suggests that a positive screening test may have led to increased clinical action.
A limitation of our study is its small size and that it was conducted in 1 pilot unit. Additionally, our retrospective analysis of clinical care inhibited our ability to fully understand a patient's clinical course or retrieve missing data points. A related limitation is that we could not ascertain how often the screening tool did not identify a case of sepsis before it was clinically diagnosed. Assessing the temporal performance of our screening tool is of great interest and may be more easily performed using an electronic version of the screening tool, which is currently in development.
Using ICD‐9 codes to determine the true‐negative cohort is another limitation of our study. It is well documented that use of administrative data can lead to inaccurate classification of patients.[14] To address this, we performed random audits of 30 test‐negative patients. In doing so we did not find any errors in classification.
Although we did not find a significant difference in screening tool performance between surgical and medical patients, the PPV of the tool was lower in the surgical population (48%) compared to the medical population (70%). The lower PPV observed in surgical patients could be attributable to an overall lower incidence of sepsis in this cohort as well as possible errors in initial assessment of infection, which can be difficult in postsurgical patients. Our retrospective analysis included data from the early months of the screening protocol, a time in which nursing staff was still developing clinical acumen in identifying sepsis. However, this could have led nurses to either overestimate or underestimate the presence of infection in either patient group.
Suspicion for infection is the cornerstone definition of sepsis, and in our screening protocol nurses were charged with making this decision based on their knowledge of the patient's clinical course and current status. Issues concerning nurses' recognition of infection symptoms are an area of opportunity for further research and education and could aid in improving PPV. Clinical judgment could be further bolstered by adding promising laboratory tests such as C‐reactive protein or procalcitonin as objective adjuncts to an initial assessment for sepsis,[15] which could potentially increase screening test PPV.
CONCLUSIONS
A simple screening tool for sepsis performed by the bedside nurse can provide a means to successfully identify sepsis early and lead to more timely diagnostics and treatment in both medical and surgical patients in an intermediate care setting.
ACKNOWLEDGEMENTS
The authors thank Eileen Pummer, quality manager for the sepsis team; Pauline Regner, patient care manager of the pilot study unit; and the nurses who contributed to the screening tool design team and data collection. The authors acknowledge Pooja Loftus for her statistical expertise, and Isabella Chu for her review of the manuscript. Disclosures: Presented as a poster at the 31st Annual Meeting of the Surgical Infection Society, Palm Beach, Florida, May 2011. The authors report no conflicts of interest.
Sepsis remains a significant healthcare burden and is the sixth most common reason for hospitalization in the United States. For patients presenting with severe sepsis, mortality rates are approximately 30%,[1, 2] and sepsis remains the most expensive reason for hospitalization. In 2009, septicemia accounted for nearly $15.4 billion in aggregate hospital costs.[2]
Early identification of sepsis and the timely implementation of goal‐directed therapy significantly decrease sepsis‐related mortality and are cost‐effective,[3, 4, 5] highlighting the need for new clinical strategies to aid in early diagnosis. To date, most studies have focused on the screening and management of sepsis in the emergency department and intensive care unit (ICU),[6, 7] and less is known about the benefits of screening in non‐ICU settings. In the non‐ICU setting, conditions may go unrecognized and treatments delayed. Evidence suggests that patients diagnosed with severe sepsis in the non‐ICU setting are almost twice as likely to die as those diagnosed in an emergency department.[8, 9]
Application of a sepsis screening tool to both medical and surgical patients poses an additional challenge that may impact the screen's performance. The specificity may be compromised by noninfectious causes of systemic inflammatory response syndrome (SIRS) commonly seen in the postsurgical patient. For example, the tachycardia and fever often seen in the postoperative patient are sufficient to qualify for SIRS, making the diagnosis of sepsis more challenging. The purpose of this study was to examine the performance of a nurse‐driven, simple sepsis screening tool in a mixed medical and surgical non‐ICU setting.
METHODS
Setting
This was an observational pilot study of prospectively screened patients admitted to a 26‐bed medical/surgical intermediate care unit with telemetry monitoring in a 613‐bed university tertiary referral hospital over a 1‐month time period. The surgical patient population of this floor consisted of cardiothoracic (50%), general (24%), and vascular surgery (17%) patients as well as a small number of trauma (7%) patients. The medical patient population admitted to this unit included pretransplant and complex medical patients requiring telemetry monitoring. Though the incidence of sepsis specific to this unit was unknown prior to the study, after an analysis of discharges the study team surmised there would be sufficient volume for testing of a nurse‐based screening tool.
Nurse Education
Registered nurses (RNs) working on the study unit had an average of 5 to 7 years of experience. The all‐RN unit was staffed predominantly at a 1:3 RN to patient ratio. RNs were supported by a clinical nurse specialist (CNS) and clinical educator (CE) RN who provided regular ongoing education about infection prevention and identification of common conditions that are seen on the unit.
In the 6 months prior to our sepsis screening initiative, nursing staff had been given more than 8 hours of education on infection‐ and sepsis‐related topics in 15‐ to 20‐minute blocks of time. This dedicated education took place during the nurses' shift in groups of 2 to 3, and was run by the CNS, assistant nurse manager, and CE RN. Nurses were also encouraged to attend an optional 8‐hour sepsis continuing medical education (CME) program. Approximately 20% of the nurses on the study unit attended.
Just prior to the pilot study, nursing staff completed a 1‐hour refresher self‐study module on severe sepsis stressing the importance of early identification. There was also a training month prior to the actual data collection time frame, where unit core trainers (RNs) or champions who had attended the optional 8‐hour sepsis CME conducted 1:1 follow‐up with each RN, reviewing at least 1 of their screens to validate understanding of screening concepts. Each RN was checked off after correctly completing a screen. During the study, unit educators and the CNS provided additional on‐unit in‐service training with screening tool completion instructions and advice on how to incorporate the tool into the RN's current assessment workflow. In addition, the charge nurses were asked to review the screens collected each shift and validate any that may have seemed inconsistent with the RN's verbal report of the patient's status.
The university's institutional review board notice of determination waived review for this study because it was classified as quality improvement.
Screening Tool
A sepsis screening tool was developed as part of a broader initiative to improve sepsis‐related morbidity and mortality at our hospital. The screening tool was adapted from the severe sepsis screening tool created by the Surviving Sepsis Campaign and Institute for Healthcare Improvement,[10] and consisted of a simple 3‐tiered paper‐based screening assessment that was to be completed by the bedside RN (Figure 1). RNs on the pilot medical/surgical intermediate care unit performed the screening assessment with their regular patient assessment at the beginning of each shift.
The first tier of the tool screened for the presence of SIRS. Positive parameters included heart rate >90, temperature >38C or <36C, white blood cell count >12,000 or<4000 or >10% bands, and/or respiratory rate >20 or partial pressure of carbon dioxide (PaCO2) <32 mm Hg. To decrease the number of false‐positive screens in patients whose abnormal vitals could already be attributed to a condition other than sepsis, these symptoms were only scored if they had emerged within the previous 8 hours.
If patients met 2 SIRS criteria, the nurse would move to the second tier of the tool, which involved consideration of possible infection as a contributor to a patient's clinical condition as well as a source of infection. If infection was not suspected, further screening was terminated. If infection was suspected, the patient then met criteria for a positive sepsis screen, and a third tier of screening involving assessment of organ dysfunction was initiated.
If the patient screened positive for sepsis (2 SIRS and suspicion for new infection) or severe sepsis (sepsis with end‐organ dysfunction), nurses were instructed to document this in the patient's electronic medical record (EMR) and call the primary team to initiate actions following the hospital‐wide sepsis guidelines. Any subsequent actions were recorded in the patient's EMR.
Data Collection
Completed sepsis screening forms during the month of October 2010 were reviewed by the authors (E.G., L.S., and P.M.). Data including age, gender, International Classification of Diseases, Ninth Revision (ICD‐9) admission and discharge diagnoses, vital signs, lab results, clinical interventions, and documented clinical decision processes by healthcare staff were collected on patients with a positive screen or those who did not screen positive but had an ICD‐9 code for sepsis, severe sepsis, or septic shock during their hospitalization or at discharge. We also collected demographic and clinical data for a random sample of patients who consistently screened negative for sepsis.
Performance Measurement
The sensitivity and specificity of the screening tool was determined by identifying true‐positive, false‐positive, true‐negative, and false‐negative results and calculating accordingly using a 2 2 contingency table. True positives were defined as cases where patients screened positive for sepsis and had a documented diagnosis of sepsis in their EMR within 24 hours of the positive screening or had an ICD‐9 billing code for sepsis. False‐positive cases were those in which patients screened positive for sepsis but did not have a diagnosis of sepsis by manual chart review nor was there an ICD‐9 code for sepsis for their hospital stay. True‐negative cases were those where patients screened negative and did not have an ICD‐9 code for sepsis. False negatives were cases where patients consistently screened negative for sepsis but had an ICD‐9 code for sepsis.
Clinical Activities
To examine the impact of a positive sepsis screen on subsequent clinical action, we assessed the frequency with which a treatment or diagnostic workup was initiated after a positive screen and compared this to clinical activity initiated after a negative screen. Specifically, the patient's EMR was reviewed for actions including measurement of lactate, blood cultures, administration of broad spectrum antibiotics, administration of fluid boluses, or consultation with or transfer to the ICU. These actions were chosen because they are part of the Surviving Sepsis Bundle, which has been demonstrated to improve mortality rates after diagnosis of severe sepsis or septic shock,[11, 12] and can be done outside of an ICU setting. Because screening was done every 8 hours, clinical activity was only attributed to a positive or negative sepsis screen if it occurred within 8 hours of the screening result. Patients were excluded if there were missing data points that precluded full analysis of their clinical course.
Statistical Analysis
To compare the performance of the screening tool between surgical and medical patients, we calculated 95% confidence intervals of screening test sensitivity and specificity. To test if performance was significantly different between these groups, we performed a nonparametric, 2‐sided, 2‐sample test of proportions. Though similar to a [2] test, the 2‐sided test of proportions allowed us to determine if there was a directional difference in test performance (ie, Does the screening tool perform better or worse in a certain patient group?). We also used the test of proportions to compare differences in the proportion of patients receiving sepsis‐related interventions before and after a positive or negative screening result. For comparisons of demographic variables we used nonparametric tests including the [2] test for categorical variables and the Kruskal‐Wallis test for continuous variables. We used SAS 9.3 (SAS Institute Inc., Cary, NC) to perform our analyses.
RESULTS
Over a 1‐month time period, 2143 screens were completed on 245 patients (169 surgical, 76 medical). The overall incidence of sepsis on the treatment unit during this time period was 9%. Surgical patients had an 8.9% incidence of sepsis, and medical patients had an incidence of 9.2%.
Screening tool performance is presented in Table 1. The screening tool had 95.5% sensitivity and 91.9% specificity, with no significant differences in performance between surgical and medical patients. The overall negative predictive value was 99.5%, also with comparable performance in both surgical and medical patients (P = 0.89). The overall positive predictive value (PPV) was 70% in medical patients and 48% in surgical patients (P = 0.12). Screening tool accuracy for medical and surgical patients was 92%.
| Overall, N = 245 (95% CI) | Surgery, N = 169 (95% CI) | Medicine, N = 76 (95% CI) | P Value* | |
|---|---|---|---|---|
| ||||
| Sensitivity | 95.5% (75%‐99.7%) | 93% (66%‐99.6%) | 100% (56%‐100%) | 0.17 |
| Specificity | 91.9% (87%‐95%) | 90% (84%‐94%) | 95% (87%‐99%) | 0.48 |
| NPV | 99.5% (81%‐100%) | 99.3% (71%‐100%) | 100% (67%‐100%) | 0.89 |
| PPV | 53.8% (39%‐70%) | 48% (23%‐73%) | 70% (30%‐100%) | 0.12 |
| LR+ | 11.8 | 9.3 | 20 | |
| LR | 0.05 | 0.08 | 0 | |
| Confirmed patient diagnosis, overall | ||||
| Sepsis | No sepsis | |||
| Screen positive | 21 (TP) | 18 (FP) | ||
| Screen negative | 1 (FN) | 205 (TN) | ||
| Confirmed patient diagnosis, medicine | ||||
| Sepsis | No sepsis | |||
| Screen positive | 7 (TP) | 3 (FP) | ||
| Screen negative | 0 (FN) | 66 (TN) | ||
| Confirmed patient diagnosis, surgery | ||||
| Sepsis | No sepsis | |||
| Screen positive | 14 (TP) | 15 (FP) | ||
| Screen negative | 1 (FN) | 139 (TN) | ||
Clinical Activities
Of the 39 patients who screened positive for sepsis, nurses classified 20 with sepsis and 19 with severe sepsis. Of these 39 patients, 33 were included in our descriptive analysis of the effect of positive screening results on clinical activity (3 were excluded for admission for sepsis and 3 for missing data). As a comparison, we randomly selected 30 patients of the 206 patients who screened negative for sepsis to evaluate clinical activity before and after a negative screen.
Characteristics of patients screening positive and negative for sepsis are reported in Table 2. We found no statistically significant differences in age, sex, length of hospital stay, or mortality amongst all groups.
| Patient Characteristics | Surgery (Positive) | Medicine (Positive) | Surgery (Negative) | Medicine (Negative) | P Value |
|---|---|---|---|---|---|
| |||||
| No. | 26 | 7 | 20 | 10 | |
| Age, y, mean | 57.8 ( 16.5) | 72.4 ( 16.8) | 64.6 ( 19.4) | 63.6 ( 16.8) | 0.25 |
| % Male (no.) | 50% (13) | 57% (4) | 60% (12) | 60% (6) | 0.27 |
| Length of stay, d, median (IQR) | 9 (716.7) | 7 (5.511.5) | 11 (7.722) | 8 (421) | 0.38 |
| No. of PODs until first positive screen, d, median (IQR) | 2 (13) | N/A | N/A | N/A | |
| % Mortality (no.) | 0% | 14% (1) | 5% (1) | 10% (1) | 0.19 |
Figure 2 illustrates differences in the proportion of patients receiving a clinical action before and after a negative or positive screening test result. In the cohort of 33 patients screening positive for sepsis, clinical action after a positive screen was taken in 4 of the 7 (50%) medical patients and 11 of 26 (42%) surgical patients. In patients screening negative for sepsis we found only 1 incident in which a sepsis‐related action was taken after a negative screen. In this case the patient was admitted to the ICU within 8 hours of a negative screen, though there was no explicit documentation that sepsis was the reason for this admission.
We compared the proportion of patients receiving sepsis‐related treatment before either a negative or positive screen and found no significant difference (Table 3). We then compared the proportion of patients receiving sepsis‐related actions after a positive or negative screening test result and found that the proportion of patients receiving antibiotics, blood cultures, and lactate measurement was significantly higher for patients with a positive sepsis screening result compared to those with a negative screening result (Table 3).
| Intervention and Group | Proportion | P Value |
|---|---|---|
| ||
| Before screening test | ||
| Antibiotics | 0.066 | |
| Positive screen | 45% | |
| Negative screen | 23% | |
| Lactate | 0.837 | |
| Positive screen | 15% | |
| Negative screen | 13% | |
| Blood culture | 0.181 | |
| Positive screen | 18% | |
| Negative screen | 17% | |
| Fluid administration | 0.564 | |
| Positive screen | 6% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.337 | |
| Positive screen | 3% | |
| Negative screen | 0% | |
| After screening test | ||
| Antibiotics | 0.006 | |
| Positive screen | 58% | |
| Negative screen | 23% | |
| Lactate | 0.018 | |
| Positive screen | 36% | |
| Negative screen | 13% | |
| Blood Culture | 0.002 | |
| Positive screen | 24% | |
| Negative screen | 17% | |
| Fluid administration | 0.112 | |
| Positive screen | 24% | |
| Negative screen | 10% | |
| ICU transfer/consult | 0.175 | |
| Positive screen | 9% | |
| Negative screen | 3% | |
DISCUSSION
Improving recognition and time to treatment of sepsis in a non‐ICU setting is an important step toward decreasing sepsis‐related mortality. Lundberg and colleagues found that mortality rates for patients diagnosed with septic shock on a general ward were higher than for patients diagnosed in the ICU, even though ward patients were younger and healthier at baseline.[8] For ward patients, treatment delays were most profound in initiating vasoactive therapies, and minor delays were encountered in initiating fluid resuscitation. In their international study on the impact of early goal‐directed therapy guidelines, Levy and colleagues found that patients diagnosed with severe sepsis on the wards were almost twice as likely to die as patients diagnosed with sepsis in the emergency department.[9]
We are the first to report about an accurate nurse‐driven SIRS‐based sepsis screening protocol that is effective in the early identification of sepsis in both medical and surgical patients in an intermediate care setting. We found no significant difference in the screening tool performance between the medical and surgical cohorts. This is an important comparison given that SIRS criteria alone can be nonspecific in the postoperative population, where it is common to have hemodynamic changes, elevation of inflammatory markers, and fevers from noninfectious sources.
Our sepsis screening tool was designed in 3 tiers to improve its specificity. The first tier was based strictly on SIRS criteria (eg, tachycardia or fever), whereas the second and third tiers served to increase the specificity of the screening tool by instructing the evaluator to assess possible sources of infection and assess for objective signs of organ dysfunction. We relied heavily on the nursing staff to assess for the presence or absence of infection and believe that the educational component prior to initiating the screening protocol was vital.
EMR‐based screening tools that rely purely on physiologic data have been considered for the early detection and management of sepsis, although they lack the specificity gained through the incorporation of clinical judgment.[13] Sawyer and colleagues report using a real‐time EMR‐based method for early sepsis detection in non‐ICU patients that is based solely on objective measures; however, their PPV was only 19.5%. The model we describe in this study is one that incorporates real‐time physiologic data available from an EMR coupled with the clinical judgment of a bedside registered nurse. As our data suggest, this provides a screen that is both sensitive and specific.
It is interesting to note that in our assessment of clinical action taken 8 hours after a positive screening test (the interval after which a new screening test was performed), the rate of diagnostic workup and/or treatment for sepsis was relatively low. One reason for this could have been that the treating team had suspicion for sepsis prior to a positive screen and had already initiated clinical action. Of the 51 recorded clinical actions taken around the time of a positive screen, the majority (67%) occurred before the screening result. It is also possible that clinical action was not pursued because the treatment team disagreed with a diagnosis of sepsis. Of all the false positive screening cases, manual chart review confirmed that these patients did not have sepsis, nor did they develop sepsis during their index hospital stay. Last, we only recorded clinical actions taken within 8 hours of the first positive screen for sepsis and measured 5 very specific actions. Thus, our analysis may have missed actions taken after 8 hours or actions that differed from the 5 we chose to assess.
Even with the apparently low levels of new clinical activity after a positive screen, when compared to patients who screened negative for sepsis, a significantly higher number of patients who had a positive screen received antibiotics, had lactate measured, and had blood cultures drawn. We did not find a significant difference in the proportion of patients receiving a sepsis‐related clinical action before a screening result (positive or negative), which suggests that a positive screening test may have led to increased clinical action.
A limitation of our study is its small size and that it was conducted in 1 pilot unit. Additionally, our retrospective analysis of clinical care inhibited our ability to fully understand a patient's clinical course or retrieve missing data points. A related limitation is that we could not ascertain how often the screening tool did not identify a case of sepsis before it was clinically diagnosed. Assessing the temporal performance of our screening tool is of great interest and may be more easily performed using an electronic version of the screening tool, which is currently in development.
Using ICD‐9 codes to determine the true‐negative cohort is another limitation of our study. It is well documented that use of administrative data can lead to inaccurate classification of patients.[14] To address this, we performed random audits of 30 test‐negative patients. In doing so we did not find any errors in classification.
Although we did not find a significant difference in screening tool performance between surgical and medical patients, the PPV of the tool was lower in the surgical population (48%) compared to the medical population (70%). The lower PPV observed in surgical patients could be attributable to an overall lower incidence of sepsis in this cohort as well as possible errors in initial assessment of infection, which can be difficult in postsurgical patients. Our retrospective analysis included data from the early months of the screening protocol, a time in which nursing staff was still developing clinical acumen in identifying sepsis. However, this could have led nurses to either overestimate or underestimate the presence of infection in either patient group.
Suspicion for infection is the cornerstone definition of sepsis, and in our screening protocol nurses were charged with making this decision based on their knowledge of the patient's clinical course and current status. Issues concerning nurses' recognition of infection symptoms are an area of opportunity for further research and education and could aid in improving PPV. Clinical judgment could be further bolstered by adding promising laboratory tests such as C‐reactive protein or procalcitonin as objective adjuncts to an initial assessment for sepsis,[15] which could potentially increase screening test PPV.
CONCLUSIONS
A simple screening tool for sepsis performed by the bedside nurse can provide a means to successfully identify sepsis early and lead to more timely diagnostics and treatment in both medical and surgical patients in an intermediate care setting.
ACKNOWLEDGEMENTS
The authors thank Eileen Pummer, quality manager for the sepsis team; Pauline Regner, patient care manager of the pilot study unit; and the nurses who contributed to the screening tool design team and data collection. The authors acknowledge Pooja Loftus for her statistical expertise, and Isabella Chu for her review of the manuscript. Disclosures: Presented as a poster at the 31st Annual Meeting of the Surgical Infection Society, Palm Beach, Florida, May 2011. The authors report no conflicts of interest.
- , , , , , Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
- , , Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. Agency for Healthcare Research and Quality. Available at: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb122.pdf. Published October 2011. Accessed on September 4, 2012.
- , , , Economic implications of an evidence‐based sepsis protocol: can we improve outcomes and lower costs? Crit Care Med. 2007;35(5):1257–1262.
- , , , , , Late compliance with the sepsis resuscitation bundle: impact on mortality. Shock. 2011;36(6):542–547.
- , , , , , The costs and cost‐effectiveness of an integrated sepsis treatment protocol. Crit Care Med. 2008;36(4):1168–1174.
- , , A simple prediction algorithm for bacteraemia in patients with acute febrile illness. QJM. 2005;98(11):813–820.
- , , , et al. Validation of a screening tool for the early identification of sepsis. J Trauma. 2009;66(6):1539–1546; discussion 1546–1547.
- , , , et al. Septic shock: an analysis of outcomes for patients with onset on hospital wards versus intensive care units. Crit Care Med. 1998;26(6):1020–1024.
- , , , et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
- Institute of Healthcare Improvement. Evaluation for severe sepsis screening tool. Surviving Sepsis Campaign. Available at: http://www.survivingsepsis.org/About_the_Campaign/Documents/evaluationforseveresepsisscreeningtool.pdf. Accessed on September 30, 2012.
- , , , et al. Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: results of a three‐year follow‐up quasi‐experimental study. Crit Care Med. 2010;38(4):1036–1043.
- , , , et al. Reduction of the severe sepsis or septic shock associated mortality by reinforcement of the recommendations bundle: a multicenter study. Ann Fr Anesth Reanim. 2010;29(9):621–628.
- , , , et al. Implementation of a real‐time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011;39(3):469–473.
- , , , Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual. 2005;20(6):319–328.
- , , , , , Comparison of procalcitonin and C‐reactive protein as markers of sepsis. Crit Care Med. 2003;31(6):1737–1741.
- , , , , , Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303–1310.
- , , Septicemia in U.S. hospitals, 2009. HCUP statistical brief #122. Agency for Healthcare Research and Quality. Available at: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb122.pdf. Published October 2011. Accessed on September 4, 2012.
- , , , Economic implications of an evidence‐based sepsis protocol: can we improve outcomes and lower costs? Crit Care Med. 2007;35(5):1257–1262.
- , , , , , Late compliance with the sepsis resuscitation bundle: impact on mortality. Shock. 2011;36(6):542–547.
- , , , , , The costs and cost‐effectiveness of an integrated sepsis treatment protocol. Crit Care Med. 2008;36(4):1168–1174.
- , , A simple prediction algorithm for bacteraemia in patients with acute febrile illness. QJM. 2005;98(11):813–820.
- , , , et al. Validation of a screening tool for the early identification of sepsis. J Trauma. 2009;66(6):1539–1546; discussion 1546–1547.
- , , , et al. Septic shock: an analysis of outcomes for patients with onset on hospital wards versus intensive care units. Crit Care Med. 1998;26(6):1020–1024.
- , , , et al. The Surviving Sepsis Campaign: results of an international guideline‐based performance improvement program targeting severe sepsis. Crit Care Med. 2010;38(2):367–374.
- Institute of Healthcare Improvement. Evaluation for severe sepsis screening tool. Surviving Sepsis Campaign. Available at: http://www.survivingsepsis.org/About_the_Campaign/Documents/evaluationforseveresepsisscreeningtool.pdf. Accessed on September 30, 2012.
- , , , et al. Impact of the Surviving Sepsis Campaign protocols on hospital length of stay and mortality in septic shock patients: results of a three‐year follow‐up quasi‐experimental study. Crit Care Med. 2010;38(4):1036–1043.
- , , , et al. Reduction of the severe sepsis or septic shock associated mortality by reinforcement of the recommendations bundle: a multicenter study. Ann Fr Anesth Reanim. 2010;29(9):621–628.
- , , , et al. Implementation of a real‐time computerized sepsis alert in nonintensive care unit patients. Crit Care Med. 2011;39(3):469–473.
- , , , Accuracy of administrative data for identifying patients with pneumonia. Am J Med Qual. 2005;20(6):319–328.
- , , , , , Comparison of procalcitonin and C‐reactive protein as markers of sepsis. Crit Care Med. 2003;31(6):1737–1741.
© 2014 Society of Hospital Medicine
