What are we to do about other factors that may affect mortality independently of smoking (e.g., diet), but which are not found in our dataset? Unfortunately, nothing. Since we do not have that information, we cannot adjust for it. In this case, diet is in statistical terms an unmeasured confounder. Unfortunately, in all observational studies there is always at least some degree of unmeasured confounding, because there may be many factors that can influence the outcome (and the exposure) which are not part of the dataset. While some statistical tools have been developed to estimate unmeasured confounding, and therefore interpret the results in its light, unmeasured confounding remains one of the major limitations of observational studies.4
Randomized, controlled trials (RCTs) on the other side do not have this problem in theory. With properly designed RCTs, all confounders, both measured and unmeasured, will be balanced between the two groups. For example, imagine an RCT where some patients are randomized to take drug A or drug B. Because patients are randomly allocated to one group or the other, it is assumed that all other factors are also randomly distributed. Hence, the two groups should be equal to each other with respect to all other factors except our active intervention, namely the type of drug they are taking (A or B). For this reason, in RCTs there is no need to adjust for multiple factors with a multivariable regression analysis, and crude unadjusted results can be presented as unbiased.
There is however a caveat. What happens if one patient who was randomized to take drug A takes drug B instead? Should she still be counted in analysis under drug A (as randomized) or under drug B (as she took it)? The usual practice is to do this and present both. In the first case, we will have the intention-to-treat (ITT) analysis, and in the second case, the per-protocol analysis (PPA). The advantage of the ITT is that it keeps the strength of randomization, namely the balancing of confounders, and therefore can present unbiased results. The advantage of the PPA is that it measures what was actually done in reality. However, in this case there is a departure from the original randomization, and hence there is the possibility of introducing confounding, because now patients are not randomly allocated to one treatment or the other. The larger the departure from randomization, the more probable the introduction of bias/confounding. For example, what if patients with more severe disease took drug A, even though they were randomized to take drug B? That will have an influence the outcome. For this reason, outcomes of the ITT analysis are considered the main results of RCTs, because PPA results can be confounded.
In summary, when reading studies, do not simply accept the results as they are presented, but rather ask yourself: “Could they be confounded by other factors, and therefore be unreliable? What steps did the authors take to reduce confounding? If they presented only crude analyses, and this was not justified by a RCT design, do they recognize it as a major limitation?” There are many nuances in every paper that can be appreciated only through a careful reading of the methods section. Hopefully, this article can shed some light on these issues and help the readers to not be confounded.
References
1. The P value: What to make of it? A simple guide for the uninitiated. GI and Hepatology News. 2019 Sep 23. https://www.mdedge.com/gihepnews/article/208601/mixed-topics/p-value-what-make-it-simple-guide-uninitiated
2. VanderWeele TJ et al. Ann Stat. 2013 Feb;41(1):196-220.
3. Concato J et al. Ann Intern Med. 1993 Feb 1;118(3):201-10.
4. VanderWeele TJ et al. Ann Intern Med. 2017 Aug 15;167(4):268-74.
Dr. Jovani is a therapeutic endoscopy fellow in the division of gastroenterology and hepatology at Johns Hopkins Hospital, Baltimore.
