ORLANDO – Studies with small patient populations but large effect sizes are the backbone of an independent investigator’s success. Rigorous patient selection doesn’t hurt, either.
“We often hide behind the words ‘pilot and feasibility’ to justify what was not a very good study,” Dr. Joshua Korzenik, director of Harvard Medical School’s Crohn’s and Colitis Center, Boston, said at a conference on inflammatory bowel diseases sponsored by the Crohn’s and Colitis Foundation of America. “The term can indicate something was not statistically significant, and that can be legitimate, but ‘pilot’ should not be a substitute for not sizing the study appropriately.”
Dr. Joshua Korzenik
Sample size consideration is important with respect to data analysis and endpoints, said Dr. Korzenik, but disciplined selection criteria strictly applied sweetens the odds for a study’s impact. Cultivating a cohort that is the “most homogeneous, cleanest, and clearest ... will give you the best insight.” Consider choosing patients according to disease subtype, bio- and genetic markers, a history of at least 3 consecutive months of disease, and a history of certain medication failures.
Steer clear of the assumption that just because you already treat a certain number of patients, you will be able to recruit them. “Some patients won’t want to commit to a study,” warned Dr. Korzenik. “You need to think more carefully.”
And don’t forget the “tremendous” impact of standard deviation on sample size. Dr. Korzenik recommended the “usual” power of .8 with a P value less than .05 for early phase studies.
For the neophyte independent investigator, sweating over what to write in his or her hypothesis, and struggling against temptation to justify sample size by stretching how small the placebo response will be vs. how great the efficacy rate is only to find actual results are not nearly what was predicted, can be devastating. “Then you’ve shot yourself in the foot,” said Dr. Korzenik.
One problem is that few, if any, previous data exist for these kinds of studies. And preclinical data “tends not to be helpful at all,” Dr. Korzenik opined.
Even in trials for anti–tumor necrosis factor drugs, what Dr. Korzenik argued are the most revolutionary treatments to yet impact the field, the question of placebo effect on sample size was tricky. “For the most part, anti-TNFs are about 20% better than placebo for inducing remission. That’s a pretty high bar to set, and most investigator-sponsored studies set the bar even higher, making it very difficult.”
If, for example, an investigator hopes to achieve a 50% reduction in calprotectin, and so sets a “modest” rate of 20% for placebo and 35% for the test drug, that means the investigator must recruit 136 patients per arm. “Yikes!”
But estimating at 15% vs. 40% for the drug, with 47 in each arm, may push the benefit of the study drug “too much.” Using a placebo effect size of 10% vs. 50% for the drug, with 17 patients per arm, the investigator runs the risk of overestimating what’s possible. “You might need to look for another endpoint, or some other set of collaborators,” Dr. Korzenik said.
Open-label studies can be useful for helping with sample size, particularly if the study is to evaluate a novel approach to treatment, but things can still go wobbly. “Open-label trials have limitations we don’t fully understand,” Dr. Korzenik said.
To wit, open-label trials on the use of the helminth Trichuris suis to treat Crohn’s disease showed robust response remission rates, but a successive, placebo-controlled trial did not achieve these results. For independent investigators conducting a placebo-controlled trial using a comparator for the control group, Dr. Korzenik suggested ways to keep the placebo response lower. These included, among other strategies, recruiting patients with higher disease activity and keeping trials as short as possible. “When you do longer studies, the placebo response remission rates go up. Keep that in mind.”
And, don’t forget: Not all small studies with impact need focus on pharmaceuticals. Possibilities Dr. Korzenik suggested include alternative interventions such as marijuana, curcumin, and aloe vera. “These things have been done, but deserve further study,” he said, adding that nutritional interventions are “undervalued, and although difficult to study, are very important.”
The role of depression, fatigue, and other psychosocial impacts of inflammatory bowel disease are also worthy of study, as are the utility of telemedicine and social media for helping patients, he said.
Because investigators will want to protect their resources – namely, the goodwill of the patients they painstakingly recruited – Dr. Korzenik advised using telemedicine to interact with study participants whenever possible, and to consider using smartphone apps to record symptom data. “Remember that repeated evaluations become an enormous burden on the patient.”