“Knowledge comes, but wisdom lingers. It may not be difficult to store up in the mind a vast quantity of facts within a comparatively short time, but the ability to form judgments requires the severe discipline of hard work and the tempering heat of experience and maturity.” – Calvin Coolidge
Dr. Chris Notte and Dr. Neil Skolnik
The information we use every day in patient care comes from one of two sources: personal experience (our own or that of another clinician) or a research study. Up until a hundred years ago, medicine was primarily a trade in which more experienced clinicians passed along their wisdom to younger clinicians, teaching them the things that they had learned during their long and difficult careers. Knowledge accrued slowly, influenced by the biased observations of each practicing doctor. People tended to remember their successes or unusual outcomes more than their failures or ordinary outcomes. Eventually, doctors realized that their knowledge base would be more accurate and accumulate more efficiently if they looked at the outcomes of many patients given the same treatment. Thus, the observational trial emerged.
As promising and important as the dawn of observational research was, it quickly became apparent that these trials had important limitations. The most notable limitations were the potential for bias and confounding variables to influence the results. Bias occurs when the opinion of the researcher influences how the result is interpreted. Confounders occur when an outcome is generated by some unexpected aspect of the patient, environment, or medication, rather than the thing that is being studied. An example of this might be a study that finds a higher mortality rate in a city by the sea than a city located inland. From these results, one might initially conclude that the sea is unhealthy. After realizing more retired people lived in the city by the sea; however, an individual would probably change his or her mind. In this example, the older age of this city’s population would be a confounding variable that drove the increased mortality in the city by the sea.
To solve the inherent problems with observational trials, the randomized, controlled trial was developed. Our reliance on information from RCTs runs so deep that it is hard to believe that the first modern clinical trial was not reported until 1948, in a paper on streptomycin in the treatment of pulmonary tuberculosis. It followed that faith in the randomized, controlled trial reached almost religious proportions, and the belief that information that does not come from an RCT should not be relied on was held by many, until recently. Why have things changed and what does this have to do with technology?
The first is an increasing recognition that, for all of their advantages, randomized trials have one nagging but critical limitation – generalizability. Randomized trials have strict inclusion and exclusion criteria. We do not have such inclusion and exclusion criteria when we take care of patients in our offices. For example, a recent trial published in the New England Journal of Medicine (2018 Dec 4. doi: 10.1056/NEJMoa1814468.), entitled “Apixaban to prevent venous thromboembolism in patients with cancer,” concluded that apixaban therapy resulted in a lower rate of venous thromboembolism than did placebo in patients starting chemotherapy for cancer. This was a large trial with more than 500 patients enrolled, and it reached an important conclusion with significant clinical implications. When you look at the details of the article, more than 1,800 patients were assessed to find the 500 patients who were eventually included in the trial. This is fairly typical of clinical trials and raises an important point: We need to be careful about how well the results of these clinical trials can be generalized to apply to the patient in front of us. This leads us to the second development that is something happening behind the scenes that each of us has contributed to.