User login
As the rigor of COVID-19 research comes under increasing scrutiny, a deep dive into contemporary trials of invasive cardiovascular interventions finds intricate ties with industry and the art of spin on full display.
After examining 216 randomized, controlled trials published in the past decade, researchers found that more than half (53.2%) were commercially funded. In 18.3% of these trials, the sponsor was involved with the trial conduct and reporting.
Commercially sponsored trials were significantly more likely to report results that favored the experimental therapy than trials without commercial sponsorship (64.3% vs. 48.5%; P = .02).
The association remained statistically significant after adjustment for differences in trial characteristics (exponent of regression coefficient beta, 2.80; 95% confidence interval, 1.09-7.18; P = .03), the authors reported in JAMA Internal Medicine.
“To make this clear, this is not an attack on industry-sponsored trials,” study author and cardiac surgeon Mario Gaudino, MD, of New York–Presbyterian and Weill Cornell Medical Center, New York, said in an interview. “Because industry has more money, they have the best trialists, the best research organization. So they generally do a pretty good trial; they’re larger, they have a higher Fragility Index, which means they’re more solid.
“And, most importantly, more than half of the trials were sponsored by industry,” he said. “So without industry, there wouldn’t be half the research in that 10-year period we explored.”
Previous research in cardiology and in other fields has shown that trials supported by for-profit organizations are more likely to report positive findings. The explanations often focus on bias and differential quality in how the trials were designed and reported.
In the present analysis, however, the authors found no difference between trials with and without industry funding in terms of estimated treatment effect, length of follow-up, use of composite or clinically significant outcomes, or outcome modification, compared with the published protocol.
Part of the explanation may be that industry-sponsored trials more often used a noninferiority design (26.1% vs. 14.9%) and had a higher loss of patients to follow-up (median of sample, 1.0% vs. 0.1%), Dr. Gaudino said. “But I think more, in general, it’s not so much a difference in the measurable characteristics of the trial. It’s the selection of the sites that participate, the patient population that is targeted that makes the trial very likely to get the result that industry would like to see.”
“Just think of the differences in the transcatheter MitraClip results between MITRA-FR and COAPT – basically they were related to the fact they enrolled different patients,” he said.
Significant spin
The analysis included 216 coronary, vascular, and structural interventional cardiology and vascular and cardiac surgical randomized, controlled trials published from January 2008 to May 31, 2019. Most were multicenter trials (78.7%); 58% originated from Europe, 12% from North America, and 10.6% from Asia.
One in six trials (16.2%) were not prospectively registered before the start of enrollment, and at least one major discrepancy existed between the registered and published primary outcome in 38% of registered trials.
“If you don’t register the trial then you can make all the changes you want to the protocol up until the moment you publish,” Dr. Gaudino observed. “There really is no rational justification for not registering a trial.”
Overall, the trials were not particularly robust, he noted. In 62 trials in which the Fragility Index was measured, only a median of five patients experiencing a different outcome in a commercially sponsored trial would change statistically significant results to nonsignificant. For noncommercially sponsored trials, that number was 4.5 and in four trials; the change in condition of only one patient was needed to switch the statistical significance.
“This finding is concerning given the substantial role that [randomized, controlled trials] results play in federal device approvals, payer criteria, and clinical consensus guidelines,” the authors wrote.
The authors also looked for interpretation bias in the trials. In the 84 trials with nonsignificant differences in the primary outcomes, 65.5% contained spin, such as focusing on statistically significant secondary outcomes or interpreting nonsignificant primary outcomes as showing treatment equivalence or comparable effectiveness. Spin was present in 80.6% of the trials with commercial sponsorship and in 54.2% without (P = .02) – a finding that remained significant after trial differences were controlled for (beta, 4.64; 95% CI, 1.05-20.54; P = .04).
A pivot point
“It’s just another paper showing there are issues with conflicts of interest in industry trials. I’m not particularly surprised,” said David Moher, PhD, MSc, director of the Centre for Journalology, based at the Ottawa Hospital Research Institute.
“It’s sort of high time people from all sides sat down together and tried to resolve how to actually move forward with industry wanting to do trials,” he said. “They are hugely important in drug development. How can these trials be done where the impact of industry and, for that matter, academia is minimized?”
Dr. Gaudino suggested the “ideal situation” would be to have industry put its funding into an existing funding organization, such as the National Institutes of Health or a newly created independent organization – a concept that has been floated before without much forward movement.
“We may be at a pivot point,” Dr. Moher said. “It’s quite clear that COVID has indicated some serious problems with how trials are done, how they’re disseminated, the notion of open science. I think this could be an opportunity. Whether there is so much noise, whether anybody will be able to take any of these initiative forward, I don’t know.”
No matter how trial funding is revised, patients must be brought to the table, he said.
“What frustrates me quite a bit is this almost parental view of all of this – the scientists know best, industry knows best,” Dr. Moher said. “We actually need the most important groups: patients and the public. They need to have an enormous amount of say in how this actually is formed.”
Commenting further, Dr. Moher said that “industry and academia can only do trials when they have patients willing to participate, and yet in the discussions you and I are having, what do patients think about spin in trials? I would imagine they would be horrified that they are going into studies – in a sense in many cases risking their lives – and yet people are spinning the results.”
Dr. Gaudino and Dr. Moher reported having no relevant conflicts of interest.
A version of this story originally appeared on Medscape.com.
As the rigor of COVID-19 research comes under increasing scrutiny, a deep dive into contemporary trials of invasive cardiovascular interventions finds intricate ties with industry and the art of spin on full display.
After examining 216 randomized, controlled trials published in the past decade, researchers found that more than half (53.2%) were commercially funded. In 18.3% of these trials, the sponsor was involved with the trial conduct and reporting.
Commercially sponsored trials were significantly more likely to report results that favored the experimental therapy than trials without commercial sponsorship (64.3% vs. 48.5%; P = .02).
The association remained statistically significant after adjustment for differences in trial characteristics (exponent of regression coefficient beta, 2.80; 95% confidence interval, 1.09-7.18; P = .03), the authors reported in JAMA Internal Medicine.
“To make this clear, this is not an attack on industry-sponsored trials,” study author and cardiac surgeon Mario Gaudino, MD, of New York–Presbyterian and Weill Cornell Medical Center, New York, said in an interview. “Because industry has more money, they have the best trialists, the best research organization. So they generally do a pretty good trial; they’re larger, they have a higher Fragility Index, which means they’re more solid.
“And, most importantly, more than half of the trials were sponsored by industry,” he said. “So without industry, there wouldn’t be half the research in that 10-year period we explored.”
Previous research in cardiology and in other fields has shown that trials supported by for-profit organizations are more likely to report positive findings. The explanations often focus on bias and differential quality in how the trials were designed and reported.
In the present analysis, however, the authors found no difference between trials with and without industry funding in terms of estimated treatment effect, length of follow-up, use of composite or clinically significant outcomes, or outcome modification, compared with the published protocol.
Part of the explanation may be that industry-sponsored trials more often used a noninferiority design (26.1% vs. 14.9%) and had a higher loss of patients to follow-up (median of sample, 1.0% vs. 0.1%), Dr. Gaudino said. “But I think more, in general, it’s not so much a difference in the measurable characteristics of the trial. It’s the selection of the sites that participate, the patient population that is targeted that makes the trial very likely to get the result that industry would like to see.”
“Just think of the differences in the transcatheter MitraClip results between MITRA-FR and COAPT – basically they were related to the fact they enrolled different patients,” he said.
Significant spin
The analysis included 216 coronary, vascular, and structural interventional cardiology and vascular and cardiac surgical randomized, controlled trials published from January 2008 to May 31, 2019. Most were multicenter trials (78.7%); 58% originated from Europe, 12% from North America, and 10.6% from Asia.
One in six trials (16.2%) were not prospectively registered before the start of enrollment, and at least one major discrepancy existed between the registered and published primary outcome in 38% of registered trials.
“If you don’t register the trial then you can make all the changes you want to the protocol up until the moment you publish,” Dr. Gaudino observed. “There really is no rational justification for not registering a trial.”
Overall, the trials were not particularly robust, he noted. In 62 trials in which the Fragility Index was measured, only a median of five patients experiencing a different outcome in a commercially sponsored trial would change statistically significant results to nonsignificant. For noncommercially sponsored trials, that number was 4.5 and in four trials; the change in condition of only one patient was needed to switch the statistical significance.
“This finding is concerning given the substantial role that [randomized, controlled trials] results play in federal device approvals, payer criteria, and clinical consensus guidelines,” the authors wrote.
The authors also looked for interpretation bias in the trials. In the 84 trials with nonsignificant differences in the primary outcomes, 65.5% contained spin, such as focusing on statistically significant secondary outcomes or interpreting nonsignificant primary outcomes as showing treatment equivalence or comparable effectiveness. Spin was present in 80.6% of the trials with commercial sponsorship and in 54.2% without (P = .02) – a finding that remained significant after trial differences were controlled for (beta, 4.64; 95% CI, 1.05-20.54; P = .04).
A pivot point
“It’s just another paper showing there are issues with conflicts of interest in industry trials. I’m not particularly surprised,” said David Moher, PhD, MSc, director of the Centre for Journalology, based at the Ottawa Hospital Research Institute.
“It’s sort of high time people from all sides sat down together and tried to resolve how to actually move forward with industry wanting to do trials,” he said. “They are hugely important in drug development. How can these trials be done where the impact of industry and, for that matter, academia is minimized?”
Dr. Gaudino suggested the “ideal situation” would be to have industry put its funding into an existing funding organization, such as the National Institutes of Health or a newly created independent organization – a concept that has been floated before without much forward movement.
“We may be at a pivot point,” Dr. Moher said. “It’s quite clear that COVID has indicated some serious problems with how trials are done, how they’re disseminated, the notion of open science. I think this could be an opportunity. Whether there is so much noise, whether anybody will be able to take any of these initiative forward, I don’t know.”
No matter how trial funding is revised, patients must be brought to the table, he said.
“What frustrates me quite a bit is this almost parental view of all of this – the scientists know best, industry knows best,” Dr. Moher said. “We actually need the most important groups: patients and the public. They need to have an enormous amount of say in how this actually is formed.”
Commenting further, Dr. Moher said that “industry and academia can only do trials when they have patients willing to participate, and yet in the discussions you and I are having, what do patients think about spin in trials? I would imagine they would be horrified that they are going into studies – in a sense in many cases risking their lives – and yet people are spinning the results.”
Dr. Gaudino and Dr. Moher reported having no relevant conflicts of interest.
A version of this story originally appeared on Medscape.com.
As the rigor of COVID-19 research comes under increasing scrutiny, a deep dive into contemporary trials of invasive cardiovascular interventions finds intricate ties with industry and the art of spin on full display.
After examining 216 randomized, controlled trials published in the past decade, researchers found that more than half (53.2%) were commercially funded. In 18.3% of these trials, the sponsor was involved with the trial conduct and reporting.
Commercially sponsored trials were significantly more likely to report results that favored the experimental therapy than trials without commercial sponsorship (64.3% vs. 48.5%; P = .02).
The association remained statistically significant after adjustment for differences in trial characteristics (exponent of regression coefficient beta, 2.80; 95% confidence interval, 1.09-7.18; P = .03), the authors reported in JAMA Internal Medicine.
“To make this clear, this is not an attack on industry-sponsored trials,” study author and cardiac surgeon Mario Gaudino, MD, of New York–Presbyterian and Weill Cornell Medical Center, New York, said in an interview. “Because industry has more money, they have the best trialists, the best research organization. So they generally do a pretty good trial; they’re larger, they have a higher Fragility Index, which means they’re more solid.
“And, most importantly, more than half of the trials were sponsored by industry,” he said. “So without industry, there wouldn’t be half the research in that 10-year period we explored.”
Previous research in cardiology and in other fields has shown that trials supported by for-profit organizations are more likely to report positive findings. The explanations often focus on bias and differential quality in how the trials were designed and reported.
In the present analysis, however, the authors found no difference between trials with and without industry funding in terms of estimated treatment effect, length of follow-up, use of composite or clinically significant outcomes, or outcome modification, compared with the published protocol.
Part of the explanation may be that industry-sponsored trials more often used a noninferiority design (26.1% vs. 14.9%) and had a higher loss of patients to follow-up (median of sample, 1.0% vs. 0.1%), Dr. Gaudino said. “But I think more, in general, it’s not so much a difference in the measurable characteristics of the trial. It’s the selection of the sites that participate, the patient population that is targeted that makes the trial very likely to get the result that industry would like to see.”
“Just think of the differences in the transcatheter MitraClip results between MITRA-FR and COAPT – basically they were related to the fact they enrolled different patients,” he said.
Significant spin
The analysis included 216 coronary, vascular, and structural interventional cardiology and vascular and cardiac surgical randomized, controlled trials published from January 2008 to May 31, 2019. Most were multicenter trials (78.7%); 58% originated from Europe, 12% from North America, and 10.6% from Asia.
One in six trials (16.2%) were not prospectively registered before the start of enrollment, and at least one major discrepancy existed between the registered and published primary outcome in 38% of registered trials.
“If you don’t register the trial then you can make all the changes you want to the protocol up until the moment you publish,” Dr. Gaudino observed. “There really is no rational justification for not registering a trial.”
Overall, the trials were not particularly robust, he noted. In 62 trials in which the Fragility Index was measured, only a median of five patients experiencing a different outcome in a commercially sponsored trial would change statistically significant results to nonsignificant. For noncommercially sponsored trials, that number was 4.5 and in four trials; the change in condition of only one patient was needed to switch the statistical significance.
“This finding is concerning given the substantial role that [randomized, controlled trials] results play in federal device approvals, payer criteria, and clinical consensus guidelines,” the authors wrote.
The authors also looked for interpretation bias in the trials. In the 84 trials with nonsignificant differences in the primary outcomes, 65.5% contained spin, such as focusing on statistically significant secondary outcomes or interpreting nonsignificant primary outcomes as showing treatment equivalence or comparable effectiveness. Spin was present in 80.6% of the trials with commercial sponsorship and in 54.2% without (P = .02) – a finding that remained significant after trial differences were controlled for (beta, 4.64; 95% CI, 1.05-20.54; P = .04).
A pivot point
“It’s just another paper showing there are issues with conflicts of interest in industry trials. I’m not particularly surprised,” said David Moher, PhD, MSc, director of the Centre for Journalology, based at the Ottawa Hospital Research Institute.
“It’s sort of high time people from all sides sat down together and tried to resolve how to actually move forward with industry wanting to do trials,” he said. “They are hugely important in drug development. How can these trials be done where the impact of industry and, for that matter, academia is minimized?”
Dr. Gaudino suggested the “ideal situation” would be to have industry put its funding into an existing funding organization, such as the National Institutes of Health or a newly created independent organization – a concept that has been floated before without much forward movement.
“We may be at a pivot point,” Dr. Moher said. “It’s quite clear that COVID has indicated some serious problems with how trials are done, how they’re disseminated, the notion of open science. I think this could be an opportunity. Whether there is so much noise, whether anybody will be able to take any of these initiative forward, I don’t know.”
No matter how trial funding is revised, patients must be brought to the table, he said.
“What frustrates me quite a bit is this almost parental view of all of this – the scientists know best, industry knows best,” Dr. Moher said. “We actually need the most important groups: patients and the public. They need to have an enormous amount of say in how this actually is formed.”
Commenting further, Dr. Moher said that “industry and academia can only do trials when they have patients willing to participate, and yet in the discussions you and I are having, what do patients think about spin in trials? I would imagine they would be horrified that they are going into studies – in a sense in many cases risking their lives – and yet people are spinning the results.”
Dr. Gaudino and Dr. Moher reported having no relevant conflicts of interest.
A version of this story originally appeared on Medscape.com.