Clinical trials produce a mountain of data that can be difficult to interpret and apply to clinical practice. When reading about studies such as the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) for schizophrenia, you may wonder:
- How large is the effect being measured?
- Is it clinically important?
- Are we dealing with a result that may be statistically significant but irrelevant for day-to-day patient care?
Number needed to treat (NNT) and number needed to harm (NNH)—two tools of evidence-based medicine (EBM, Box 11,2)—can help answer these questions. This article shows how to calculate NNT and NNH, then applies these tools to published results from CATIE phases 1 and 2.
What does ‘evidence-based’ mean?
Evidence-based medicine (EBM) is a process by which a clinician extracts information from the medical literature and applies it in day-to-day patient treatment. Gray and Pinson1 summarize EBM’s 5 steps as:
- formulate the question
- search for answers
- appraise the evidence
- apply the results
- assess the outcome.
This is not a trivial task. To help clinicians, EBM pioneers such as Gordon Guyatt, MD, MSc, and Drummond Rennie, MD, have published useful, readable, short reviews of EBM methods in the “Users’ Guides to the Medical Literature” in the Journal of the American Medical Association.2
Internet resources also are available, including:
- Centre for Evidence-Based Medicine, University of Toronto. www.cebm.utoronto.ca
- Eskind Biomedical Library, Vanderbilt University. Evidence-based knowledge portal. www.mc.vanderbilt.edu/biolib/ebmportal/login.html
- Hayward Medical Communications. Evidence-Based Medicine: What is…? series. www.evidence-based-medicine.co.uk/What_is_series.html.
What is nnt?
NNT helps us gauge effect size—or clinical significance. It is different from knowing if a clinical trial result is statistically significant.
NNT allows us to place a number on how often we can expect to see a difference between two interventions. If we see a therapeutic difference once every 100 patients (an NNT of 100), the difference between two treatments is not of great concern under most circumstances. But if a difference in outcome is seen once in every 5 patients being treated with one intervention versus another (an NNT of5), the result will likely influence day-to-day practice. Together with calculating a confidence interval (Box 2),3 the NNT can help you judge the clinical significance of a statistically significant result.
Use confidence intervals to determine if NNT is statistically significant
NNT is useful when examining differences in binary outcomes such as treatment response (yes/no), remission (yes/no), or avoidance of hospitalization (yes/no). NNT also is useful when we compare two medications’ side effects. Under these circumstances, we call NNT the “number needed to harm” (NNH).Calculating number needed to treat (NNT) or number needed to harm (NNH) does not tell you whether the result is statistically significant. You can find out by examining a range of values called the confidence interval (CI).
An NNT with a 95% CI means that the truth probably lies between the lower and upper bounds of the interval with a probability of 95%. A 95% CI with an NNT of 5 to 15 means we have an NNT that with 95% certainty falls between 5 and 15.
Formulas can be used to calculate CIs.3 One useful online calculator is available at: www.cebm.utoronto.ca/practise/ca/statscal.
Sometimes the lower bound of a CI is a negative number and the upper bound is a positive number (such as –10 to +10). This occurs when the result is not statistically significant. Having a negative number and a positive number in the CI means when comparing intervention A to intervention B, intervention A might be better than B, or B might be better than A. We could not conclude that a difference exists between the two interventions.
Calculating nnt and nnh
NNT and NNH are easy to calculate:
- First determine the difference between the frequencies of the outcome of interest for two interventions.
- Then calculate the reciprocal of this difference.
- Difference in response rates=0.75–0.55=0.20
- NNT=1/0.20=5.
Interpreting the importance of NNT values is easy, too. The smaller the NNT, the larger the clinical difference between interventions; the larger the NNT, the smaller the difference.