There are three main ways to deal with this problem. The first is to have only one main outcome declaring statistical significance for only that outcome and consider the other outcomes as exploratory. The second is to report on multiple findings and correct for multiple testing. The third is to report on multiple findings, but mention explicitly in the paper that they have not corrected for multiple testing and therefore the findings may be significant by chance.
Conclusion
In summary, the P value is the probability of a false-positive finding, and the cutoff of .05 is arbitrary. Instead of dichotomizing results as “significant” and “nonsignificant” purely based on whether the P value is more or less than .05, a more qualitative approach that takes into account the magnitude of the P value and the sample size should be considered, and multiple testing should be taken into account when declaring significant findings.
Dr. Jovani is a therapeutic endoscopy fellow, division of gastroenterology and hepatology, Johns Hopkins Hospital, Baltimore.
References
1. Guyatt G et al. CMAJ. 1995;152:27-32.
2. Guller U and DeLong ER. J Am Coll Surg. 2004;198:441-58.
3. Greenland S et al. Eur J Epidemiol. 2016;31:337-50.
