Studies often include subgroup analyses outlining how a specific treatment is more or less effective in one group of patients compared with another. But clinicians, beware: Subgroup analyses too often are not clinically meaningful and should be interpreted cautiously, Dr. Sarah R. Barton and her associates reported in a poster presentation at the American Society for Clinical Oncology’s Gastrointestinal Cancers Symposium.
The investigators reviewed 145 randomized, controlled phase III trials published in peer-reviewed journals from January 2003 to January 2012 that tested an investigational therapy in GI cancer and that involved at least 150 patients. Subgroup analyses appeared in 100 studies (69%), more often in larger ones.
Courtesy of Wikimedia Commons user Mate2Code (Creative Commons)Boolean functions illustrate subsets
Here’s the shocking part: Only 25% of trials that claimed the treatment worked in a subgroup of patients had the statistical measures to back that up, reported Dr. Barton of Royal Marsden Hospital, Sutton, England. That proportion was the same for industry-sponsored and nonindustry trials.
The study, which won a Merit Award at the meeting, conducted some interesting subgroup analyses of its own. Trials sponsored by for-profit companies included a significantly higher number of subgroup analyses compared with nonindustry trials – a median of six versus two, respectively.
Trials of targeted therapies were more than three times as likely to report subgroup analyses compared with studies of cytotoxic therapies and included significantly more subgroup analyses (a median of six vs. two, respectively). Studies that reported a positive effect in the primary outcome also included a significantly higher median number of subgroup analyses compared with negative trials (again, six versus two).
Industry-sponsored trials that reported a positive effect in the primary outcome of the study were the most likely to report subgroup analyses (23 of 25 studies, or 95%) and to include the highest median number of subgroup analyses (eight) compared with industry-funded trials with a negative primary outcome or nonindustry trials, positive or negative.
Dr. Barton gave some clues that, in general, should cause physicians to look closely at efficacy claims. These include subgroup analyses conducted post hoc, when multiple tests are applied, when multiple endpoints are used, and if there’s no statistically significant test of interaction.
This is not just a problem in oncology. A previous study of 469 randomized, controlled trials published in 118 journals reported that industry-funded trials were less likely to define subgroups before starting the trial, less likely to use the interaction test for analyses of subgroup effects, and more likely to report on subgroups if the primary outcome in the study did not show a positive result (BMJ 2011;342:d1569)
The New England Journal of Medicine provides similar cautions in its guidelines for investigators reporting on subgroup analyses (N. Engl. J. Med. 2007;357:2189-2194).
Dr. Barton reported having no financial disclosures.
– Sherry Boschert
On Twitter @sherryboschert