| MDedge

Article Type

Article

Changed

Fri, 12/21/2018 - 07:18

Display Headline

Playing by the Rules: Using Decision Rules Wisely Part 2, Nontraumatic Conditions

Author(s)

Emmagene Worley, MD

James Mattson, MD

Biren A. Bhatt, MD

In this second part of “Playing by the Rules,” we will examine validated clinical decision rules that assist emergency physicians (EPs) in the diagnosis and treatment of nontraumatic conditions. Most trauma rules seek to answer a yes or no question regarding the utility of testing for specific disease states when the diagnosis is not clinically apparent.

For example, the Canadian CT Head Rule describes a number of conditions that, if met, can predict the absence of traumatic lesions requiring neurosurgical intervention in the alert patient with head injury, and thus obviate the need for imaging in those instances. In contrast, many medical rules are actually risk stratification scales for treatment and diagnosis, categorizing patients into low- to high-risk groups based on clinical factors. While traumatic conditions are linked to a specific inciting event or “trauma,” medical diseases may have multiple causative factors or may be delayed in presentation to the emergency department (ED), which subsequently increases the complexity of these decision instruments.

Rather than an exhaustive list of all clinical decision rules or risk stratification scales relevant to emergency medicine, this installment will provide EPs with a review of common instruments and the evidence behind them.

Central Nervous System

Ottawa Subarachnoid Hemorrhage Rule

The Ottawa Subarachnoid Hemorrhage Rule offers guidance for diagnosing atraumatic subarachnoid hemorrhage (SAH) in alert, neurologically intact adult patients presenting to the ED with a headache reaching maximal intensity within 1 hour of onset. The rule states that if none of the following conditions are present, then the diagnosis of SAH can be excluded without further testing:

Symptom of neck pain or stiffness
Age greater than 40 years old
Witnessed loss of consciousness
Onset during exertion
Thunderclap headache with peak pain instantly
Limited neck flexion on exam

The validation study prospectively enrolled 1153 adults of whom 67 had a positive workup for SAH (defined as subarachnoid blood visible on noncontrast CT scan of the head, xanthochromia of cerebrospinal fluid on visual inspection, or the presence of >1 million erythrocytes in the final tube of cerebrospinal fluid with an aneurysm or arteriovenous malformation confirmed on cerebral angiography).¹ Of note, patients with prior history of cerebral aneurysm or SAH were excluded, as were patients with recurrent headaches similar to the presenting complaint, patients with focal neurologic deficits or papilledema, or patients with a history of brain neoplasm, ventricular shunt, or hydrocephalus. The authors found that the rule was 100% sensitive and 13% specific for detecting SAH, with a kappa of 0.82, which suggests good interrater reliability.¹

Comment: It is important to note that the authors excluded patients with a history of cerebral aneurysm or prior SAH, and therefore the rule should not be applied to these patients in clinical practice. The utility of this rule is somewhat limited secondary to the age cutoff, as the incidence of aneurysmal SAH increases considerably after the fifth decade of life.² Ultimately, this rule—combined with the authors’ previous work showing that CT performed within 6 hours of headache onset can rule out SAH—provides a powerful diagnostic tool for EPs considering SAH in the ED.³

ABCD2 Score

The ABCD2 score was developed to identify transient ischemic attack (TIA) patients at risk for early stroke, and thus inform decisions regarding admission and resource utilization in the ED and outpatient clinic setting.4 The score was created by combining elements of two previously existing rules, the California and the ABCD scales. Patients presenting with TIA symptoms are assigned points based on:

Age: 1 point if ≥ 60 years
Blood Pressure: 1 point if ≥ 140/90
Clinical Deficit: 2 points for unilateral weakness, 1 point for speech impairment without unilateral weakness
Duration: 2 points for ≥ 60 minutes, 1 point for 10 to 59 minutes
Diabetes: 1 point if diabetic

The greater the number of points, the higher the risk for imminent stroke, from low (0-3 points) to moderate (4-5 points) to high (6-7 points). The initial retrospective internal validation study found that the low, moderate, and high groups correlated to 7-day stroke risk of 1.2%, 5.9%, and 11.7%, respectively. Subsequently, the ABCD2 score was rapidly incorporated into institutional and national protocols for assessing risk for stroke and featured prominently in the 2009 American Heart Association guidelines on TIA, which recommend hospitalization for a score of 3 or greater.^4,5

More recently, a multicenter prospective external validation study of more than 2000 TIA patients found that using the American Heart Association recommended cutoff of 3 or greater resulted in a sensitivity of 94.7% for detecting those patients who sustained a stroke within 7 days, but a specificity of only 12.5%.⁶ The investigators concluded that a specificity this low would require “almost all” of the TIA patients in their cohort (87.6%) to be admitted to the hospital—even though only 3.2% of their patients had a stroke within 90 days.⁶ Even when examined at other cutoff scores, the investigators found the ABCD2 score to have poor accuracy.⁶

Comment: Decreasing resource utilization is a laudable goal, but it does not appear that the ABCD2 score provides much guidance on which TIA patients can safely go home. Moreover, the increasing availability of advanced imaging and tele-neurology consultation in the ED have changed the landscape of TIA and stroke care. Many EPs have since argued that the ABCD2 score adds little to their evaluation.⁷

Abdomen

Alvarado Score

There are multiple clinical prediction rules for appendicitis. Among the most commonly utilized by EPs and surgical consultants are the Alvarado score and the Appendicitis Inflammatory Response Score. The Alvarado score was derived in 1986 based on a retrospective review of 305 abdominal pain patients of whom 227 (aged 4 to 80 years) had appendicitis.8 Factors were identified and weighted, which can be recalled through the mnemonic MANTRELS:

Migration of pain to the right lower quadrant: 1 point
Anorexia or acetone in urine: 1 point
Nausea or vomiting: 1 point
Tenderness in the right lower quadrant: 2 points
Rebound tenderness: 1 point
Elevation of the temperature > 37.3°C: 1 point
Leukocytosis >10K X 109/L: 2 points
Shift to the left of neutrophils (>75%): 1 point

The original article posited that a score of 5 or 6 was “compatible” with the diagnosis of acute appendicitis—necessitating further observation for possible appendicitis—and that higher scores indicated an increasing probability of disease.⁸ Of note, the rule has also been adapted for clinical settings where differentials are not easily obtainable with the left shift criterion removed; this is known as the modified Alvarado score and calculated out at a maximum of 9.⁹

Since the original Alvarado study was published, multiple small studies have attempted to validate or otherwise retrospectively assess the utility of this rule. A frequently cited systematic review of 42 prospective and retrospective studies by Ohle et al found that a score of <5 showed a sensitivity of 99% overall (96% in men, 99% in women, and 99% in children) for ruling out admission/observation of patient with suspected appendicitis, though the specificity for ruling in the diagnosis at scores 7 and higher was only 81% overall.¹⁰

However, a more recent prospective observational study of adult abdominal pain patients presenting to large American urban EDs found the modified Alvarado rule at cutoff levels of 3, 4, and 5 had sensitivities of only 72%, 55%, and 36%, respectively, of ruling out the diagnosis.¹¹ In comparison, the study found that physicians’ clinical judgement of appendicitis being the first or second most likely diagnosis had a sensitivity of 93% for predicting appendicitis.¹¹

Comment: The Alvarado score was developed to help rule out and rule in the diagnosis of appendicitis. However, with the increasing availability of CT scanning in EDs, the diagnostic pathway in unclear cases has shifted from admission/observation to CT scanning, which has the benefit of elucidating other pathology as well. The utility of the Alvarado rule has been called into question. Ultimately, there is data in support of the Alvarado rule from older articles and studies in resource-poor environments, and newer studies may reflect less rigorous application of the rule when CT scanning is the default clinical pathway. Further studies that focus specifically on the Alvarado score as a rule out test to decrease CT utilization may be instructive.

Appendicitis Inflammatory Response (AIR) score

The appendicitis inflammatory response (AIR) score was derived in a cohort of 316 patients and validated on a sample of 229 adults and children with suspected appendicitis.12 The authors specifically sought to create a rule that outperformed the Alvarado score; the criteria are:

Vomiting: 1 point
Right iliac fossa pain: 1 point
Rebound tenderness: 1 point for light, 2 for medium, 3 for strong
Temperature >38.5°C: 1 point
Polymorphonuclear leukocytes: 1 point for 70%-84%, 2 for 85% or greater
White blood cell count: 1 point for 10,000-14,900, 2 for 15,000 or greater
C-reactive protein level (mg/dL): 1 point for 10-49, 2 for 50 or greater

Patients with a score of 0-4 were classified as low risk, with recommendation for outpatient follow-up if general condition unchanged; a score of 5-8 as indeterminate risk, with recommendation for active observation with serial exams, imaging, or diagnostic laparoscopy; or a score of 9-12 as high risk, with recommendation for surgical exploration.¹² In the validation cohort, the investigators found an AIR score or Alvarado score greater than 4 to have, respectively, 96% or 97% sensitivity and 73% or 61% specificity for detecting appendicitis.¹² A high score of greater than 8 on either the AIR or Alvarado had respectively 37% or 28% sensitivity but specificity of 99% for detecting appendicitis with either instrument.¹²

In an external validation study, the AIR and Alvarado scores were calculated on a series of 941 patients (aged 1 to 97 years) being evaluated for possible appendicitis; 201 patients were younger than 18.¹³ At a cutoff of greater than 4, the sensitivity and specificity were found to be 93% and 85% for the AIR and 90% and 55% for Alvarado.¹³ In a cohort of 182 patients (aged 4 to 75 years), a score of 4 or greater on the AIR and Alvarado was found to have comparable sensitivity to that of a senior surgical consultant for detecting appendicitis—with sensitivities of 94%, 93%, and 90% respectively.¹⁴ Subsequently, the original investigators undertook a large multicenter implementation study of the AIR at 24 hospitals of patients (aged 5 to 96 years) with suspected appendicitis. As compared to the pre-implementation group, using AIR to categorize patients as low risk resulted in significantly fewer imaging studies, admissions, and surgical explorations.¹⁵

Comment: The AIR has the benefit of recent prospective studies that assess performance of the rule in settings that mirror the practice environments of most EPs today. The classification of rebound tenderness as light, medium, or strong may be difficult to ascertain. Ultimately, reductions in imaging, admissions, and surgical explorations are important goals and EPs might benefit from using this rule to guide imaging.

CHEST

HEART Score

The increasingly popular HEART score, first developed by physicians in the Netherlands in 2008, seeks to risk-stratify patients presenting to the ED with suspected cardiac chest pain without ST-elevation myocardial infarction (STEMI). It scores patients 0 to 2 on 5 different characteristics (with a total scored of 10 possible points):

History: 2 points for highly suspicious, 1 point for moderately suspicious
EKG: 2 points for significant ST deviation, 1 point for nonspecific repolarization disturbance
Age: 2 points for age 65 years or greater, 1 point for age 45-64 years
Risk Factors: 2 points for 3 or more risk factors or history of atherosclerotic disease, 1 point for 1 to 2 risk factors
Troponin: 2 points for troponin value >3 times the normal limit, 1 point for value 1-3 times the normal limit.

The authors developed these 5 categories “based on clinical experience and current medical literature,” and then applied the rule to 122 chest pain patients in the ED, finding a higher incidence of major adverse coronary events (MACE) with increasing score: 2.5% for low risk score of 0-3, 20.3% for intermediate risk score of 4-6, and 72.7% for score 7 or higher.¹⁶ The score has been retrospectively and prospectively validated.^17,18 In a study of 2440 patients, the low risk group had a MACE of 1.7%, and the score had a c-statistic of 0.83, outperforming Thrombolysis in Myocardial Infarction (TIMI) and GRACE c-statistics of 0.75 and 0.70, respectively.¹⁸ In 2013, investigators calculated the HEART score on a multinational database of 2906 chest pain patients, finding a negative predictive value of 98.3% for MACE with HEART score less than or equal to 3.¹⁹

In the United States, Mahler et al have produced a series of 3 articles validating the HEART score and demonstrating its use in reducing cardiac testing and length of stay. In 1070 patients admitted to their observation unit, who were deemed low risk by physician assessment and TIMI <2, a score of less than or equal to 3 had a negative predictive value of 99.4% for MACE; the inclusion of serial troponins resulted in sensitivity of 100%, specificity of 83.1%, and negative predictive value of 100%.²⁰ The team then conducted a secondary analysis of chest pain patients enrolled in a large multicenter trial (MIDAS) and compared HEART score, the North American Chest Pain Rule, and unstructured clinical assessment.²¹ Both rules had high sensitivities, but the HEART score identified 20% of patients suitable for early discharge, as compared to 4% for the North American Chest Pain Rule.²¹ Finally, Mahler’s team performed a randomized control trial of 282 patients investigating whether the HEART score with serial troponins compared with usual care could safely reduce cardiac testing.²² The HEART pathway resulted in an absolute reduction of 12.1% in cardiac testing, and median reduction in length of stay by 12 hours, with no missed MACE in discharged patients.²²

Most recently, a stepped-wedge, cluster randomized trial across 9 hospitals published in 2017 investigated the utility of the HEART score. Despite enrolling only 3648 patients out of the statistically required sample size of 6600, they found that the HEART score was not inferior to usual care and there was no significant difference in median length of stay, but health care resources were typically lower in the HEART score group.²³

Comment: While derived in a less conventional manner, the HEART score has held up in several validation studies and appears poised to safely decrease health care costs and increase ED efficiency and throughput. As more US EDs look to adopt high sensitivity troponin biomarkers, prospective studies will be needed to determine the role of the HEART score in this setting.

Thrombolysis in Myocardial Infarction (TIMI) score

The Thrombolysis in Myocardial Infarction (TIMI) score was developed in 2000 as a tool to risk-stratify patients with a diagnosis of unstable angina (UA) and non–ST-elevation myocardial infarction (NSTEMI). The score was derived from 1 arm (2047 patients) of a study comparing heparin with enoxaparin for treatment of NSTEMI, and validated in the other 3 arms of the study (5124 patients). Multivariate logistic regression was used to develop 7 variables of equal weight:

Age greater than or equal to 65yo
Three or more cardiac risk factors
Known coronary artery disease (with stenosis greater than or equal to 50%)
Aspirin use in the past 7 days
Severe angina (2 or more episodes in the past 24 hours)
EKG ST changes greater than or equal to 0.5 mm
Positive serum cardiac biomarkers

The investigators found that with a higher score, there was progressive increase in adverse cardiac outcomes, with a c-statistic of 0.65.²⁴ This score was subsequently validated across several existing databases evaluating various therapeutic interventions for UA/NSTEMI and remained statistically significant, with increasing risk for MI and mortality with increasing score.^25,26

Given the success in predicting patient outcomes and identifying patients who could benefit from more aggressive care, the TIMI risk score was then applied to unselected ED chest pain patients. In a secondary analysis of a prospective observational cohort of 3929 patient visits, the TIMI score correlated to the risk for adverse outcomes, with a risk of 2.1% at score 0.²⁷

In a second prospective observational cohort of 1458 patient visits, a score of 0 correlated to a 1.7% incidence of adverse outcomes.²⁸ In 2008, Body et al sought to increase the relative weight of EKG and biomarker factors to 5 (instead of 1) in a study of 796 patients, positing that these factors have more importance in the ED setting.²⁹ Comparing the modified TIMI to the original, the modified instrument improved the area under curve (AUC) from 0.77 to 0.87.²⁹ In follow-up validation studies, the modified score has an improved AUC, but the incidence of adverse outcomes at score 0 remains at about 2% for both modified and original score.^30,31

In 2010, Hess et al performed a systematic review and meta-analysis of the studies that prospectively validated the TIMI score. They evaluated 10 validation studies, encompassing 17,265 patients across 5 countries, and found a strong linear relation between the TIMI score and adverse cardiac events.³² At TIMI score of 0, the incidence of cardiac events was 1.8%, with sensitivity of 97.2% and specificity of 25%. Subsequently, the ADAPT trial designed a diagnostic protocol consisting of TIMI risk assessment, EKG, and 0- and 2-hour troponin I biomarkers to find ED patients eligible for safe, early discharge.³³ Of the 1975 patients, 20% were classified as low risk and eligible for early discharge, in that they had TIMI score of 0, a non-ischemic ECG, and negative troponins. Only one patient had a MACE at 30 days, giving the protocol a sensitivity of 99.7%, specificity of 23.4%, and negative predictive value of 99.7%.³³

As the TIMI and HEART scores are both used to evaluate ED chest pain patients, several studies have sought to compare them. In 2015, Cartlon et al published a comparison of 5 established risk scores and 2 troponin assays in 963 patients: modified Goldman, TIMI, GRACE, HEART, and Vancouver Chest Pain Rule in combination with troponin T and I.³⁴ The investigators found that a negative troponin T plus either TIMI score of 0 or a HEART score ≤3 gave a negative predictive value of greater than 99.5% with more than 30% of patients able to be discharged safely.³⁴ In 2017, a comparison of the GRACE, HEART, and TIMI scores in 1833 chest pain patients found the HEART score identified more low risk patients than either of its comparators and had the highest AUC at 0.86.³⁵ Other trials have similarly found HEART outperforming TIMI.³⁶

Comment: The TIMI score was not specifically designed for ED use but has been adapted to serve this purpose. To the EP assessing the undifferentiated chest pain patient, the TIMI score uses clinical variables that may seem curious (eg, aspirin use) or impossible for EPs to ascertain (eg, presence or degree of stenosis). Even for patients with a score of 0, the risk for adverse outcomes remains stubbornly at the 2% level, similar to the original low risk HEART score findings.

Wells’ Criteria for Pulmonary Embolism

The diagnosis of pulmonary embolism (PE) is often challenging, requiring the use of multiple ED resources for timely diagnosis, and is therefore well suited for clinical decision instruments. The Wells’ Criteria were derived from a cohort of 1260 patients using logistic regression to identify 7 significant variables:

Clinical signs and symptoms of deep vein thrombosis (DVT): 3
PE is the most likely diagnosis: 3
Heart rate >100: 1.5
Immobilization or surgery in the previous 4 weeks: 1.5
Previously diagnosed DVT or PE: 1.5
Hemoptysis: 1
Malignancy with treatment within 6 months or palliative: 1

The investigators specifically linked the use of their instrument to the D-dimer assay, using their score to determine pretest probability and seeking to exclude the diagnosis in patients with low pretest probability and negative D-dimer result.^37,38 They reported a three-tiered classification, with low risk at a score less than 2, moderate risk at scores from 2-6, and high risk at scores greater than 6. The risk for PE with a low risk score coupled with a negative D-dimer result were 1.5% and 2.7% in the derivation and validation cohorts. Using a two-tiered classification of PE unlikely at scores less than or equal to 4 and PE likely at scores 5 or greater, a PE unlikely score and a negative D-dimer had a 2.2% and 1.7% risk in the derivation and validation cohorts.

A subsequent study by Wells et al on 930 ED patients using the score plus D-dimer found a negative predictive value of 99.5% for a low risk score and a negative D-dimer.³⁹ This allowed for reduced imaging in 53% of patients.³⁹ Another external validation study found acceptable interrater agreement between physicians for the Wells’ score at kappa 0.62 for the three-tiered system and 0.7 for the two-tiered system.⁴⁰ The Wells’ Criteria has been compared against the Geneva score with receiver operating characteristic curve analysis showing no difference between the two rules.⁴¹ In a large cohort of 3306 patients being evaluated for PE using the Wells’ score and D-dimer, for the 1028 patients with PE unlikely and a negative D-dimer, there was a 3-month incidence of venous thromboembolism (VTE) of 0.5%—none of which were fatal events.⁴²

Comment: The Wells’ Criteria for pulmonary embolism combined with D-dimer is now the preferred approach for many EPs seeking to risk-stratify their patients for PE. Advances in age-adjusted cutoffs for D-dimer provide additional support for this powerful tool.

Pulmonary Embolism Rule-Out Criteria (PERC)

Given the low specificity of the D-dimer assay for VTE, researchers post–Wells’ Criteria have sought to further reduce unnecessary testing by reassessing the D-dimer’s role in the diagnostic pathway. The PERC rule was designed to reduce D-dimer use—and downstream CT scan testing—in low-risk patients. The investigators derived the rule from a population of patients for whom the pretest probability of PE was less than 15%, seeking a risk for PE less than 2% if the rule was satisfied. Using logistic regression in 3148 ED patients, 8 clinical criteria were obtained:

Age < 50 years Pulse <100
Pulse oximetry >94%
No unilateral leg swelling
No hemoptysis
No recent surgery
No prior PE/DVT
No hormone use

The variables were tested in 1427 low-risk and 382 very-low-risk patients (defined as being evaluated for dyspnea, but not part of the derivation or low-risk validation groups). In the low-risk group, the sensitivity, specificity, and false-negative rate was 96%, 27%, and 1.4% respectively. In the very-low-risk group, the sensitivity, specificity, and false-negative rate was 100%, 15%, and 0% respectively.⁴³ The rule was further validated in a prospective multicenter study of 8138 patients; among patients with pretest probability less than 15% who were PERC negative, 1% had PE/DVT within 45 days.⁴⁴ The large PERCEPIC trial on 1757 patients found low pretest probability patients who were PERC negative had a false-negative rate of 1.2% and estimated that the use of PERC could decrease the median length of stay in the ED by at least 2 hours.⁴⁵ The PROPER study, a non-inferiority, crossover cluster-randomized trial in 14 EDs across France, found that use of the PERC rule was not inferior to conventional care and that it was associated with reduced ED length of stay and CT use.^45,46

There has been criticism from some European studies that the PERC rule misses too many PEs. A provocatively titled multinational study from Hugli et al examined patients suspected to have PE in Switzerland, France, and Belgium. The investigators applied the PERC rule and then stratified the patients by pretest probability as defined by the Geneva score, which includes many of the same criteria as PERC. They found the PERC rule identified a small proportion of patients with suspected PE as very low risk (13.2%) and that the prevalence of PE among these patients was 5.4%. Critics of this study have noted that the PERC rule was designed to be applied in low-risk patients, not to define the low-risk population.⁴⁷ Another study examined a retrospective cohort of patients in whom a D-dimer was ordered to exclude PE, and then calculated the Wells’ and PERC score from the medical record. The investigators found that the combination of Wells and PERC missed 2 PEs out of their population of 377 patients.⁴⁸ However, a subsequent meta-analysis analyzed 11 studies—including the two negative studies—and found a pooled sensitivity of 97%, specificity of 23%, and negative likelihood ratio of 0.18, concluding that when the pretest probability is low, PERC is sensitive enough to exclude D-dimer testing.⁴⁹

Comment: Given the number of disease states and sampling techniques that can cause nonspecific elevation in D-dimer assay, the PERC rule provides a useful tool in low-risk populations for excluding PE without laboratory testing. The key is applying the rule to the appropriate population, as stratified by gestalt or clinical score.

Infectious Disease

Mortality in Emergency Department Sepsis (MEDS) score

The Mortality in Emergency Department Sepsis (MEDS) score was developed as a risk stratification tool for patients presenting to the ED with concern for sepsis. This score was prospectively derived from a population of 3301 ED patient encounters during which a blood culture was ordered. Charts were reviewed and several data points extracted and analyzed to determine the following univariate predictors of 28-day mortality: terminal illness, tachypnea or hypoxia, septic shock, platelets <150,000/mm3, bands >5%, age >65 years, lower respiratory infection, nursing home residence, and altered mental status. These predictors were assigned point values based on their odds ratios, and points are added to generate a total score. Mortality risk was stratified into groups based on total score, with percentage mortality as follows: score 0-4: 0.9%; 5-7: 2.0%; 8-12: 7.8%; 13-15: 20.2%; >15: 50%. A separate validation cohort had the following mortality rates: score 0-4: 1.1%; 5-7: 4.4%; 8-12: 9.3%; 13-15: 16.1%; >15: 39%.⁵⁰

The MEDS score was subsequently shown to also be predictive for 1-year mortality as well, with an area under receiver operating curve (AUROC) of 0.76 for 1-year mortality.⁵¹ A subsequent study showed similar mortality rates when expanding the patient population to include all patients with systemic inflammatory response syndrome (SIRS), potentially broadening the potential application of MEDS in ED risk stratification.⁵² However, the score was shown to be less predictive in patients with severe sepsis and septic shock, underestimating mortality in all MEDS score groups.⁵³ Still, the MEDS score was demonstrated in multiple validation studies as a reliable risk stratification tool in patients with suspected infection or SIRS.^54,55

Comment: The MEDS score is not as well studied in the literature as the SIRS criteria or QuickSOFA but is a validated risk stratification tool in patients with suspected infection and is ED specific. This tool, similar to Pneumonia Severity Index and CURB-65 (discussed below), can guide management of patients from the ED. Very-low-risk (score 0-4) patients can be treated as outpatients, low risk (score 5-7) patients warranting consideration of a short inpatient stay, and moderate to high risk (>8) requiring inpatient management. At present, there is insufficient evidence regarding the role of the MEDS score to guide inpatient disposition of floor vs. ICU in moderate to high-risk patients.

Pneumonia Severity Index

The Pneumonia Severity Index (PSI) was developed as a tool to predict mortality risk from pneumonia, allowing providers to appropriately manage care for these patients in the hospital or as outpatients. A derivation cohort of 14199 patients was utilized to create a prediction rule in two steps meant to parallel a clinician’s decision-making process. The first step identified a population of patients that were at low risk for death, which were assigned to class I. The second step quantified the risk for death in the remaining patients using weighted factors including demographics, comorbidities, exam findings, and clinical data. In all, 20 variables were used and assigned corresponding points, the sum of which would assign a patient to a particular risk for mortality (class II-V).⁵⁶

Mortality risk was relatively low for patients in class I and II (0.4 and 0.7%, respectively). Class III carried a mortality risk of 2.8%. Mortality increased with class IV and class V classification: 8.5% and 31.1%, respectively. These data were replicated with a separate validation cohort of 38039 patients, with similar mortality rates in each class. This study concluded with the recommendation that patients diagnosed with pneumonia falling into class I and II mortality risk should be managed as outpatients, possible brief inpatient observation for class III, and class IV and V managed as inpatients.⁵⁶

Subsequent trials evaluating the utility of the PSI score in the management of patients diagnosed with pneumonia randomized low-risk patients (class I-III PSI) to treatment as outpatients vs inpatients. There were no statistical differences in adverse outcomes (ICU admission, hospital readmission, mortality, complications), with notable improvements in hospital admission rates and patient satisfaction.^57,58 A meta-analysis of 6 studies that used a clinical decision tools to identify low-risk patients to treat pneumonia as outpatients showed no significant difference in mortality, patient readmissions, or patient satisfaction. Low-risk patients that required admission often included comorbid illnesses not included in the PSI, inability to take oral medications, barriers to compliance, or hypoxemia.⁵⁹

Though the PSI has been shown to successfully identify patients at low risk for mortality, it has been less accurate at predicting and stratifying classes of severe pneumonia. A meta-analysis by Loke et al showed that PSI class IV or V had pooled sensitivity of 0.90 and specificity 0.53 for 30-day mortality, which was significantly better than the CURB-65 rule (discussed below).⁶⁰ However, a subsequent large meta-analysis showed that PSI class IV or V had a sensitivity of 75% and specificity 40% for requiring ICU intervention or admission, which are not sufficient to guide disposition decisions.⁶¹

CURB-65

One of the criticisms of PSI included its complexity, with inclusion of 20 factors making it impractical for use as a bedside tool. The CURB-65 score was developed with a similar goal of identifying low-risk patients with pneumonia who would be candidates for outpatient management, but also patients at high risk for mortality or ICU admission. Criteria for severe pneumonia published by the British Thoracic Society include: respiratory rate ≥ 30 breaths/min, diastolic blood pressure ≤60 mmHg, and blood urea nitrogen >7 mmol/L. The presence of 2 criteria was 88% sensitive and 72% specific for mortality or ICU admission.⁶² The CURB-65 tool was based on these criteria, with the addition of age ≥65 years, which was found to be a separate independent predictor of mortality. Thus, the 5 criteria making up the score are as follows (1 point each, 0-5 total):

Confusion, meaning Mental Test Score ≤8, or disorientation to person, place, or time
Urea >7 mmol/L (>19.6 mg/dL)
Respiratory rate ≥ 30 breaths/minute
Blood pressure (systolic < 90 mmHg or diastolic ≤ 60 mmHg)
Age ≥ 65 years

A score of 0-1 of these criteria characterized low mortality risk (<1.5%) in the test group, a score of 2 was intermediate mortality risk (9.2%), and a score of 3 or more associated with high mortality risk (22%). A score ≥ 2 was 93% sensitive and 49% specific for 30-day mortality.⁶³

A subsequent prospective validation study by Aujesky et al that included 3181 patients with community-acquired pneumonia demonstrated slightly higher mortality rates for each CURB-65 score (0.6%, 3%, 6.1%, 13%, 17%, 43% mortality in scores of 0-5, respectively).⁶⁴ In particular, the 3% mortality rate in CURB-65 scores of 1 is similar to PSI class III mortality rates, suggesting a lower threshold (CURB-65 ≥1) for consideration of admission for management. Another validation study by Capelastegui et al showed similar mortality rates to the derivation study for specific CURB-65 scores, but noted 53% of patients with a score of 1 also were found to have characteristics that were independent for a poor prognosis, and should be considered in the decision for outpatient or inpatient treatment.⁶⁵ Furthermore, a recent study found that of patients in the ED with a CURB-65 score of 1, 8% still required critical care intervention.⁶⁶ Thus, use of CURB-65 in screening for low-risk patients with community-acquired pneumonia is recommended to be limited to scores of 0. However, as with PSI, validation studies have yet to show predictive utility of scores suggesting severe pneumonia (CURB-65 ≥3) in predicting mortality or ICU requirement.^60,61

As validation studies have suggested only patients with a CURB-65 score of 0 are screened low risk enough for outpatient treatment, greater weight may be placed on utility of CRB-65 as a tool. This rule, initially proposed in the same study as CURB-65, omits blood urea nitrogen as a factor to only rely on history and physical exam data with a score of 0 indicating low risk.⁶³ In initial derivation and validation studies, this rule demonstrated <1.6% mortality risk with a score of 0, with risk increasing to 4-8.6% in scores of 1.^63,65 Multiple studies have compared CRB-65 and CURB-65, with only marginal but not statistically significant improvement in prognostic utility of CURB-65.^65,67 A meta-analysis of 1648 patients even showed only 0.5% mortality risk in CRB-65 ≤1; potentially including CRB-65 0-1 as low risk, though, would require further study.⁶⁸ Although multiple validation studies have also successfully stratified low risk to similar mortality risk (<1.6%), accuracy wanes with higher CRB-65 scores.⁶⁹

Several studies have directly compared CURB-65 and PSI both in terms of identifying low-risk patients and stratifying disease severity.^{60,61,64,68,70-72} Multiple studies have shown similar mortality risk in low-risk populations and have demonstrated sensitivities for mortality greater than 96% for CURB-65/CRB-65 = 0 and PSI class I-III, albeit with specificities ranging from 18-65%.^64,68,70 In stratifying patients into different levels of severity (ward vs ICU patients), PSI has shown slightly better sensitivity/specificity for mortality and/or ICU intervention, though neither is strong enough to significantly stratify severe pneumonia to serve as tools for directing inpatient management.^60,61

Comment: PSI, CRB-65, and CURB-65 have been well validated as screening tools for low-risk patients who should be treated as outpatients (CURB-65 or CRB-65 = 0, PSI class I and II). A moderate-risk population (CURB-65 = 1 or 2, PSI class III) may benefit from treatment as inpatient or outpatient at clinician judgement. Use of these tools for determining disease severity and possible ICU requirement is not as reliable, and other clinical factors should be considered.

Conclusion

This article provides an overview of several common clinical decision instruments and the evidence behind them. Ultimately, many institutions have incorporated clinical decision rules directly into the electronic medical record, and this strategy will not only increase their use, but hopefully collect further data on whether the instruments truly perform better than unstructured clinical judgement.

References

1. Perry JJ, Sivilotti MLA, Sutherland J, et al. Validation of the Ottawa Subarachnoid Hemorrhage Rule in patients with acute headache. CMAJ. 2017;189(45):E1379-E1385.

2. de Rooij NK, Linn FH, van der Plas JA, Algra A, Rinkel GJ. Incidence of subarachnoid haemorrhage: a systematic review with emphasis on region, age, gender and time trends. J Neurol Neurosurg Psychiatry. 2007;78(12):1365-1372.

3. Perry JJ, Stiell IG, Sivilotti ML, et al. Sensitivity of computed tomography performed within six hours of onset of headache for diagnosis of subarachnoid haemorrhage: prospective cohort study. BMJ. 2011;343:d4277.

4. Johnston SC, Rothwell PM, Nguyen-Huynh MN, et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet. 2007;369(9558):283-292.

5. Easton JD, Saver JL, Albers GW, et al. Definition and evaluation of transient ischemic attack: a scientific statement for healthcare professionals from the American Heart Association/American Stroke Association Stroke Council; Council on Cardiovascular Surgery and Anesthesia; Council on Cardiovascular Radiology and Intervention; Council on Cardiovascular Nursing; and the Interdisciplinary Council on Peripheral Vascular Disease. The American Academy of Neurology affirms the value of this statement as an educational tool for neurologists. Stroke. 2009;40:2276-2293

6. Perry JJ, Sharma M, Sivilotti ML, et al. Prospective validation of the ABCD2 score for patients in the emergency department with transient ischemic attack. CMAJ. 2011;183(10):1137-1145.

7. Stead LG, Suravaram S, Bellolio MF, et al. An assessment of the incremental value of the ABCD2 score in the emergeny department evaluation of transient ischemic attack. Ann Emerg Med. 2011;57(1):46-51.

8. Alvarado A. A practical score for the early diagnosis of acute appendicitis. Ann Emerg Med. 1986;15(5):557-564.

9. Kalan M, Talbot D, Cunliffe WJ, Rich AJ. Evaluation of the modified Alvarado score in the diagnosis of acute appendicitis: a prospective study. Ann R Coll Surg Engl. 1994;76(6):418-419.

10. Ohle R, O'Reilly F, O'Brien KK, Fahey T, Dimitrov BD. The Alvarado score for predicting acute appendicitis: a systematic review. BMC Med. 2011;9:139.

11. Meltzer AC, Baumann BM, Chen EH, Shofer FS, Mills AM. Poor sensitivity of a modified Alvarado score in adults with suspected appendicitis. Ann Emerg Med. 2013;62(2):126-31.

12. Andersson M, Andersson RE. The appendicitis inflammatory response score: a tool for the diagnosis of acute appendicitis that outperforms the Alvarado score. World J Surg. 2008;32(8):1843-1849.

13. de Castro SM, Ünlü C, Steller EP, van Wagensveld BA, Vrouenraets BC. Evaluation of the appendicitis inflammatory response score for patients with acute appendicitis. World J Surg. 2012;36(7):1540-1545.

14. Kollár D, McCartan DP, Bourke M, Cross KS, Dowdall J. Predicting acute appendicitis? A comparison of the Alvarado score, the Appendicitis Inflammatory Response Score and clinical assessment. World J Surg. 2015;39(1):104-109.

15. Andersson M, Kolodziej B, Andersson RE; STRAPPSCORE Study Group. Randomized clinical trial of Appendicitis Inflammatory Response score-based management of patients with suspected appendicitis. Br J Surg. 2017;104(11):1451-1461.

16. Six AJ, Backus BE, Kelder JC. Chest pain in the emergency room: value of the HEART score. Netherlands Heart J. 2008;16(6):191-196.

17. Backus BE, Six AJ, Kelder JC, et al. Chest Pain in the Emergency Room. A Multicenter Validation of the HEART Score. Crit Pathways Cardiol. 2010;9:164-169.

18. Backus BE, Six AJ, Kelder JC, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. In J Cardiol. 2013;168:2153-2158.

19. Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain in the emergency department. Crit Pathways Cardiol. 2013;12:121-126.

20. Mahler SA, Hiestand BC, Goff DC, Hoekstra JW, Miller CD. Can the HEART score safely reduce stress testing and cardiac imaging in patients at low risk for acute coronary syndrome? Crit Pathw Cardiol. 2011:10(3):128-133.

21. Mahler SA, Miller CD, Hollander JE, et al. Identifying patients for early discharge: performance of decision rules among patients with acute chest pain. Int J Cardiol. 2013;168(2):795-802.

22. Mahler SA, Riley RF, Hiestand BC, et al. The HEART Pathway Randomized Trial: identifying emergency department patients with acute chest pain for early discharge. Circ Cardiovasc Qual Outcomes. 2015;8(2):195-203.

23. Poldervaart JM, Reitsma JB, Backus BE, et al. Effect of using the HEART score in patients with chest pain in the emergency department: a stepped-wedge, cluster randomized trial. Ann Intern Med. 2017;166:687-697.

24. Antman EM, Cohen M, Bernink PJLM, et al. The TIMI risk score for unstable angina/non-ST eevation MI. JAMA. 2000;284:835-842.

25. Scirica BM, Cannon CP, Antman EM, et al. Validation of the Thrombolysis In Myocardial Infarction (TIMI) risk score for unstable angina pectoris and non-ST-elevation myocardial infarction in the TIMI III registry. Am J Cardiol. 2002;90:303-305.

26. Morrow DA, Antman EM, Snapinn SM, McCabe CH, Theroux P, Braunwald E. An integrated clinical approach to predicting the benefit of tirofiban in non-ST elevation acute coronary syndromes. Eur Heart J. 2002;23:223-229.

27. Pollack CV, Sites FD, Shofer FS, Sease KL, Hollander JE. Application of the TIMI risk score for unstable angina and non–ST-elevation acute coronary syndrome to an unselected emergency department chest pain population. Acad Emerg Med. 2006:13(1):13-18.

28. Chase M, Robey JL, Zogby KE, Sease KL, Shofer FS, Hollander JE. Prospective validation of the Thrombolysis in Myocardial Infarction risk score in the emergency department chest pain population. Ann Emerg Med. 2006;48(3):252-259.

29. Body R, Carley S, McDowell G, Ferguson J, Mackway-Jones K. Can a modified thrombolysis in myocardial infarction risk score outperform the original for risk stratifying emergency department patients with chest pain? Emerg Med J. 2009;26:95-99.

30. Hess EP, Perry JJ, Calder LA, et al. Prospective validation of a modified Thrombolysis In Myocardial Infarction risk score in emergency department patients with chest pain and possible acute coronary syndrome. Acad Emerg Med. 2010;17(4):368-375.

31. Macdonald SPJ, Nagree Y, Fatovich DM, ad Brown SGA. Modified TIMI risk score cannot be used to identify low-risk chest pain in the emergency department: a multicenter validation study. Emerg Med J. 2014;31:281-285.

32. Hess EP, Agarwal D, Chandra S, et al. Diagnostic accuracy of the TIMI risk score in patients with chest pain in the emergency department: a meta-analysis. CMAJ. 2010;182(10):1039-1044.

33. Than, M, Cullen L, Aldous S, et al. 2-Hour accelerated diagnostic protocol to assess patients with chest pain symptoms using contemporary troponins as the only biomarker: the ADAPT trial. JACC. 2012;59(23):2091-2098.

34. Carlton EW, Khattab A, Greaves K. Identifying patients suitable for discharge after a single-presentation high-sensitivity Troponin result: a comparison of five established risk scores and two high-sensitivity assays. Ann Emerg Med. 2015;66(6):635-645.

35. Poldervaart JM, Langedijk M, Backus BE, et al. Comparison of the GRACE, HEART and TIMI score to predict major adverse cardiac events in chest pain patients at the emergency department. Int J Cardiol. 2017;227:656-661.

36. Nieuwets A, Poldervaart JM, Reitsma JB, et al. Medical consumption compared for TIMI and HEART score in chest pain patients at the emergency department: a retrospective cost analysis. BMJ Open. 2016;6:e010694.

37. Wells PS, Ginsberg JS, Anderson DR, et al. Use of a clinical model for safe management of patients with suspected pulmonary embolism. Ann Intern Med. 1998;129:997-1005.

38. Wells PS, Anderson DR, Rodger M, et al. Derivation of a simple clinical model to categorize patients’ probability of pulmonary embolism: increasing the model’s utility with the SimpliRED D-dimer. Thromb Haemost. 2000;83(3):416-420.

39. Wells PS, Anderson DR, Rodger M, et al. Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and D-dimer. Ann Intern Med. 2001;135:98-107.

40. Wolf SJ, McCubbin TR, Feldhaus KM, Faragher JP, Adcock DM. Prospective validation of Wells’ criteria in the evaluation of patients with suspected pulmonary embolism. Ann Emerg Med. 2004;44:503-510.

41. Chagnon I, Bounameaux H, Aujesky D, et al. Comparison of two clinical prediction rules and implicit assessment among patients with suspected pulmonary embolism. Am J Med. 2002;113:269-275.

42. Christopher Study Investigators. Effectiveness of managing suspected pulmonary embolism using an algorithm combining clinical probability, D-dimer testing, and computed tomography. JAMA. 2006;295:172-179.

43. Kline JA, Mitchell AM, Kabrhel C, Richman PB, Courtney DM. Clinical criteria to prevent unnecessary diagnostic testing in emergency department patients with suspected pulmonary embolism. J Thromb Haemost. 2004;2:1247-1255.

44. Kline JA, Courtney DM, Kabrhel C, et al. Propsective multicenter evaluation of the pulmonary embolism rule-out criteria. J Thromb Haemost. 2008;6:772-780.

45. Penaloza A, Soulie C, Moumneh T, et al. Pulmonary embolism rule-out criteria (PERC) rule in European patients with low implicit clinical probability (PERCEPIC): a multicenter, prospective, observational study. Lancet Haematol. 2017;4:e615-e621.

46. Freund Y, Cachanado M, Aubry A, et al. Effect of the Pulmonary Embolism Rule-Out Criteria on subsequent thromboembolic events among low-risk emergency department patients. The PROPER randomized clinical trial. JAMA. 2018;319(6):559-566.

47. Hugli O, Righini M, Le Gal G, et al. The pulmonary embolism rule-out criteria (PERC) rule does not safely exclude pulmonary embolism. J Thromb Haemost. 2011;9:300-4.

48. Theunissen JMG, Scholing C, van Hasselt WE, van der Maten J, ter Avest E. A retrospective analysis of the combined use of PERC rule and Wells score to exclude pulmonary embolism in the Emergency Department. Emerg Med J. 2016;33:696-701.

49. Singh B, Parsaik AK, Aharwal D, Surana A, Mascarenhas SS, Chandra S. Diagnostic accuracy of Pulmonary Embolism Rule-Out Criteria: a systematic review and meta-analysis. Ann Emerg Med. 2012;59(6):517-520.

50. Shapiro NI, Wolfe RE, Moore RB, Smith E, Burdick E, Bates DW. Mortality in Emergency Department Sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med. 2003;31(3):670-675.

51. Shapiro NI, Howell MD, Talmor D, Donnino M, Ngo L, Bates DW. Mortality in Emergency Department Sepsis (MEDS) score predicts 1-year mortality. Crit Care Med. 2007;35(1):192-198.

52. Sankoff JD, Goyal M, Gaieski DF, et al. Validation of the Mortality in Emergency Department Sepsis (MEDS) score in patients with the systemic inflammatory response syndrome (SIRS). Crit Care Med. 2008;36(2):421-26.

53. Jones AE, Saak K, Kline JA. Performance of the Mortality in Emergency Department Sepsis score for predicting hospital mortality among patients with severe sepsis and septic shock. Am J Emerg Med. 2008;26(6):689-692.

54. Carpenter CR., Keim SM, Upadhye S, Nguyen HB. Risk stratification of the potentially septic patient in the emergency department: the Mortality in the Emergency Department Sepsis (MEDS) score. J Emerg Med. 2009;37(3):319-327.

55. Hermans MAW, Leffers P, Jansen LM, Keulemans YC, Stassen PM. The value of the Mortality in Emergency Department Sepsis (MEDS) score, C reactive protein and lactate in predicting 28-day mortality of sepsis in a Dutch emergency department. Emerg Med J. 2012;29(4):295–300.

56. Fine MJ, Auble TE, Yealy DM, et al. A Prediction Rule to Identify Low-Risk Patients with Community Acquired Pneumonia. N Engl J Med. 1997;326(4):243-250.

57. Marrie TJ, Lau CY, Wheeler SL, et al. A controlled trial of a critical pathway for treatment of community-acquired pneumonia. JAMA. 2000;283(6):749-755. doi:10.1001/jama.283.6.749.

58. Carratalà J, Fernandez-Sabe N. Outpatient care compared with hospitalization for community-acquired pneumonia: a randomized trial in low-risk patients . Ann Intern Med. 2005;142:165-172. doi:10.7326/0003-4819-142-3-200502010-00006.

59. Chalmers JD, Akram AR, Hill AT. Increasing outpatient treatment of mild community-acquired pneumonia: Systematic review and meta-analysis. Eur Respir J. 2011;37(4):858-864. doi:10.1183/09031936.00065610.

60. Loke YK, Kwok CS, Niruban A, Myint PK. Value of severity scales in predicting mortality from community-acquired pneumonia: systematic review and meta-analysis. Thorax. 2010;65(10):884-890. doi:10.1136/thx.2009.134072.

61. Marti C, Garin N, Grosgurin O, et al. Prediction of severe community-acquired pneumonia: A systematic review and meta-analysis. Crit Care. 2012;16(4):R141. doi:10.1186/cc11447.

62. Neill AM, Martin IR, Weir R, et al. Community-acquired pneumonia: aetiology and usefulness of severity criteria on admission. Thorax. 1996;51(10):1010-1016. doi:10.1136/thx.51.10.1010.

63. Lim WS, Van Der Eerden MM, Laing R, et al. Defining community acquired pneumonia severity on presentation to hospital: An international derivation and validation study. Thorax. 2003;58(5):377-382. doi:10.1136/thorax.58.5.377.

64. Aujesky D, Auble TE, Yealy DM, et al. Prospective comparison of three validated prediction rules for prognosis in community-acquired pneumonia. Am J Med. 2005;118(4):384-392. doi:10.1016/j.amjmed.2005.01.006.

65. Capelastegui A, España PP, Quintana JM, et al. Validation of a predictive rule for the management of community-acquired pneumonia. Eur Respir J. 2006;27(1):151-157. doi:10.1183/09031936.06.00062505.

66. Ilg A, Moskowitz A, Konanki V, et al. Performance of the CURB-65 score in predicting critical care interventions in patients admitted with community-acquired pneumonia. Ann Emerg Med. 2018. doi:10.1016/j.annemergmed.2018.06.017.

67. Bauer TT, Ewig S, Marre R, Suttorp N, Welte T. CRB-65 predicts death from community-acquired pneumonia. J Intern Med. 2006;260(1):93-101. doi:10.1111/j.1365-2796.2006.01657.x.

68. Akram AR, Chalmers JD, Hill AT. Predicting mortality with severity assessment tools in out-patients with community-acquired pneumonia. QJM. 2011;104(10):871-879. doi:10.1093/qjmed/hcr088.

69. McNally M, Curtain J, O’Brien KK, Dimitrov BD, Fahey T. Validity of British Thoracic Society guidance (the CRB-65 rule) for predicting the severity of pneumonia in general practice: Systematic review and meta-analysis. Br J Gen Pract. 2010;60(579):423-433. doi:10.3399/bjgp10X532422.

70. Shah BA, Ahmed W, Dhobi GN, Shah NN, Khursheed SQ, Haq I. Validity of Pneumonia Severity Index and CURB-65 severity scoring systems in community acquired pneumonia in an Indian Setting. Indian J Chest Dis Allied Sci. 2010;52(1):9-17.

71. Noguchi S, Yatera K, Kawanami T, et al. Pneumonia severity assessment tools for predicting mortality in patients with cealthcare-associated pneumonia: a systematic review and meta-analysis. Respiration. 2017;93(6):441-450. doi:10.1159/000470915.

72. Kolditz M, Braeken D, Ewig S, Rohde G. Severity assessment and the immediate and long-term prognosis in community-acquired pneumonia. Semin Respir Crit Care Med. 2016;37(6):886-896. doi:http://dx.doi.org/10.1055/s-0036-1592127.

Article PDF

bhatt1218.pdf

Author and Disclosure Information

Author’s Disclosure Statement: The authors report no actual or potential conflict of interest in relation to this article.

Dr. Worley is Instructor, Department of Emergency Medicine, NewYork-Presbyterian Hospital/Columbia University, New York, NY. Dr. Mattson is Resident, Department of Emergency Medicine, NewYork-Presbyterian Hospital, New York, NY. Dr. Bhatt is Assistant Professor, Department of Emergency Medicine, NewYork-Presbyterian Hospital/Columbia University, New York, NY.

Issue

Emergency Medicine - 50(12)

Publications

Emergency Medicine

MDedge Emergency Medicine

Topics

Page Number

1-14

Read more about Playing by the Rules: Using Decision Rules Wisely Part 2, Nontraumatic Conditions

Sections

Feature