User login
Conventional wisdom holds that for randomized, controlled trials, it’s all about the primary endpoint. If it’s negative or neutral, then none of the other results means much beyond “hypothesis generating.”
This strict-constructionist thinking has now been called into question. A recent article in the New England Journal of Medicine declared “an unreasonable yet widespread practice is the labeling of all randomized trials as either positive or negative on the basis of whether the P value for the primary outcome is less than .05. This view is overly simplistic.” (2016 Sept 1;375[9]:861-70).
The article, by the highly experienced and respected trialists Stuart J. Pocock, PhD, and Gregg W. Stone, MD, adds this: “If the primary outcome is negative, positive findings for secondary outcomes are usually considered to be hypothesis generating. Certainly, regulatory approval of a new drug is unlikely to follow. However, in some instances, secondary findings are compelling enough to affect guidelines and practice.”
This unconventional take from a pair of high-level trialists was especially timely given the buzz around the results from two studies reported at the European Society of Cardiology annual congress in late August, DANISH and NORSTENT.
The DANISH trial compared the impact of implantable cardioverter-defibrillators (ICDs) plus optimal care against optimal care without ICDs in 1,116 patients with nonischemic systolic heart failure. The primary outcome, all-cause death during more than 5 years of follow-up, was a relative 13% less with ICD use, a difference that was not statistically significant, and one secondary outcome, cardiovascular death, was cut by a relative 25% with ICD use, also not statistically significant.
But for the study’s second prespecified secondary endpoint of sudden cardiac death, treatment with ICDs cut the rate in half, compared with nonischemic heart failure patients who did not receive an ICD, a 4-percentage-point difference that was statistically significant.
And in a prespecified secondary analysis of the primary endpoint that broke down the study group by age, the two-thirds of patients younger than 68 years had a significant reduction in all-cause mortality with ICD use, a benefit not seen in patients aged 68 or older.
Discussion of the results at the meeting mainly focused on what meaning, if any, could be drawn from these strongly positive secondary outcomes in a trial neutral for its primary outcome.
“The ICDs did what they were supposed to, prevent sudden cardiac death,” said the lead investigator of the study, Lars Køber, MD. “As a principle I say don’t believe in a subgroup, but guidelines are often based on subgroup analyses.”
“The primary outcome was neutral, but the reduction in sudden cardiac death, the primary objective of an ICD, was significant, so an ICD should be taken into consideration,” commented Michel Komajda, MD, a discussant for the report.
After I wrote a news article about the DANISH report at ESC, I received an email from a reader who objected to spinning the results this way and insisted that no valid lessons can be drawn from the DANISH results because the study’s primary endpoint failed to show a statistical significance. This purist view misses the important, relevant lessons from the DANISH results. The DANISH trial was not designed to provide pivotal data for regulatory approval of ICDs in these patients. Rather, Dr. Køber and his associates designed DANISH to see whether ICD use in these patients could cut all-cause death over a fairly long follow-up. It was a very high bar and ICDs failed, but the deck was stacked against an ICD win. Enrolled patients averaged 64 years old at entry into the study, and they all had New York Heart Association class II or III heart failure. “The overall survival curves start to diverge, but then converge after 5 years because of the comorbidities and patients dying for other reasons,” Dr. Køber noted.
“The message is, in younger patients with less morbidity and more life expectancy, sudden cardiac death is a bigger problem, and they had a substantial drop in mortality” with ICD use, commented heart failure specialist Javed Butler, MD. “It’s very consistent with the way we think about providing ICD treatment to patients.”
In other words, the DANISH results showed that all patients with nonischemic systolic heart failure can’t expect to live substantially longer during extended follow-up if they get an ICD, because the cut in sudden cardiac death the devices provide eventually gets washed out by the many other risks for death these patients face. But younger, relatively healthier patients might very well see their reduced rate of sudden cardiac death translate into an overall mortality benefit even when they are followed for at least 5 years. That’s important information to help an individual patient decide whether to have an ICD placed, and an important message from the DANISH trial despite the neutral primary endpoint.
NORSTENT involved a similar scenario in a trial that addressed a totally different issue: Should patients with either stable or unstable coronary artery disease who are undergoing coronary stenting receive a drug-eluting stent (DES) or a bare metal stent (BMS)? The trial randomized 9,013 patients to receive either of the two stent types plus optimal medical therapy. The primary endpoint was the rate of all-cause death or nonfatal MI during 5 years of follow-up, and the results showed no statistically significant difference between the patients who received a DES and those who got a BMS (N Engl J Med. 2016 Aug 30. doi: 10.1056/NEJMoa1607991).
But for the secondary endpoint of repeat revascularizations performed during follow-up, the use of a DES cut the procedure rate by 3.3 percentage points, a 17% relative risk reduction that was statistically significant. The use of a DES also cut the stent thrombosis rate by 0.4 percentage points, a one-third relative drop in these events that was also statistically significant.
In short, despite the neutral primary endpoint for the trial, the results showed that drug-eluting stents did what they were designed to do relative to bare metal stents: cut the rate of target lesion restenosis and the need for repeat revascularization. Several interventional cardiologists who heard the results at the meeting said that the findings would not change their practice and that they would continue to use the DES as their default device for percutaneous coronary interventions. Although “the long-term benefit of contemporary DES over BMS was less than expected,” said Kaare H. Bønaa, MD, lead investigator for the NORSTENT trial, the secondary benefit of significantly reduced repeat revascularization and the very modest price difference that now exists between drug-eluting stents and bare metal stents means that many interventionalists will continue to use a DES for most patients.
The message from Dr. Pocock and Dr. Stone, underscored by the DANISH and NORSTENT results, is that large and well-run randomized trials can yield important evidence to inform practice that transcends a simple black or white statistical assessment of the primary endpoint.
On Twitter @mitchelzoler
Conventional wisdom holds that for randomized, controlled trials, it’s all about the primary endpoint. If it’s negative or neutral, then none of the other results means much beyond “hypothesis generating.”
This strict-constructionist thinking has now been called into question. A recent article in the New England Journal of Medicine declared “an unreasonable yet widespread practice is the labeling of all randomized trials as either positive or negative on the basis of whether the P value for the primary outcome is less than .05. This view is overly simplistic.” (2016 Sept 1;375[9]:861-70).
The article, by the highly experienced and respected trialists Stuart J. Pocock, PhD, and Gregg W. Stone, MD, adds this: “If the primary outcome is negative, positive findings for secondary outcomes are usually considered to be hypothesis generating. Certainly, regulatory approval of a new drug is unlikely to follow. However, in some instances, secondary findings are compelling enough to affect guidelines and practice.”
This unconventional take from a pair of high-level trialists was especially timely given the buzz around the results from two studies reported at the European Society of Cardiology annual congress in late August, DANISH and NORSTENT.
The DANISH trial compared the impact of implantable cardioverter-defibrillators (ICDs) plus optimal care against optimal care without ICDs in 1,116 patients with nonischemic systolic heart failure. The primary outcome, all-cause death during more than 5 years of follow-up, was a relative 13% less with ICD use, a difference that was not statistically significant, and one secondary outcome, cardiovascular death, was cut by a relative 25% with ICD use, also not statistically significant.
But for the study’s second prespecified secondary endpoint of sudden cardiac death, treatment with ICDs cut the rate in half, compared with nonischemic heart failure patients who did not receive an ICD, a 4-percentage-point difference that was statistically significant.
And in a prespecified secondary analysis of the primary endpoint that broke down the study group by age, the two-thirds of patients younger than 68 years had a significant reduction in all-cause mortality with ICD use, a benefit not seen in patients aged 68 or older.
Discussion of the results at the meeting mainly focused on what meaning, if any, could be drawn from these strongly positive secondary outcomes in a trial neutral for its primary outcome.
“The ICDs did what they were supposed to, prevent sudden cardiac death,” said the lead investigator of the study, Lars Køber, MD. “As a principle I say don’t believe in a subgroup, but guidelines are often based on subgroup analyses.”
“The primary outcome was neutral, but the reduction in sudden cardiac death, the primary objective of an ICD, was significant, so an ICD should be taken into consideration,” commented Michel Komajda, MD, a discussant for the report.
After I wrote a news article about the DANISH report at ESC, I received an email from a reader who objected to spinning the results this way and insisted that no valid lessons can be drawn from the DANISH results because the study’s primary endpoint failed to show a statistical significance. This purist view misses the important, relevant lessons from the DANISH results. The DANISH trial was not designed to provide pivotal data for regulatory approval of ICDs in these patients. Rather, Dr. Køber and his associates designed DANISH to see whether ICD use in these patients could cut all-cause death over a fairly long follow-up. It was a very high bar and ICDs failed, but the deck was stacked against an ICD win. Enrolled patients averaged 64 years old at entry into the study, and they all had New York Heart Association class II or III heart failure. “The overall survival curves start to diverge, but then converge after 5 years because of the comorbidities and patients dying for other reasons,” Dr. Køber noted.
“The message is, in younger patients with less morbidity and more life expectancy, sudden cardiac death is a bigger problem, and they had a substantial drop in mortality” with ICD use, commented heart failure specialist Javed Butler, MD. “It’s very consistent with the way we think about providing ICD treatment to patients.”
In other words, the DANISH results showed that all patients with nonischemic systolic heart failure can’t expect to live substantially longer during extended follow-up if they get an ICD, because the cut in sudden cardiac death the devices provide eventually gets washed out by the many other risks for death these patients face. But younger, relatively healthier patients might very well see their reduced rate of sudden cardiac death translate into an overall mortality benefit even when they are followed for at least 5 years. That’s important information to help an individual patient decide whether to have an ICD placed, and an important message from the DANISH trial despite the neutral primary endpoint.
NORSTENT involved a similar scenario in a trial that addressed a totally different issue: Should patients with either stable or unstable coronary artery disease who are undergoing coronary stenting receive a drug-eluting stent (DES) or a bare metal stent (BMS)? The trial randomized 9,013 patients to receive either of the two stent types plus optimal medical therapy. The primary endpoint was the rate of all-cause death or nonfatal MI during 5 years of follow-up, and the results showed no statistically significant difference between the patients who received a DES and those who got a BMS (N Engl J Med. 2016 Aug 30. doi: 10.1056/NEJMoa1607991).
But for the secondary endpoint of repeat revascularizations performed during follow-up, the use of a DES cut the procedure rate by 3.3 percentage points, a 17% relative risk reduction that was statistically significant. The use of a DES also cut the stent thrombosis rate by 0.4 percentage points, a one-third relative drop in these events that was also statistically significant.
In short, despite the neutral primary endpoint for the trial, the results showed that drug-eluting stents did what they were designed to do relative to bare metal stents: cut the rate of target lesion restenosis and the need for repeat revascularization. Several interventional cardiologists who heard the results at the meeting said that the findings would not change their practice and that they would continue to use the DES as their default device for percutaneous coronary interventions. Although “the long-term benefit of contemporary DES over BMS was less than expected,” said Kaare H. Bønaa, MD, lead investigator for the NORSTENT trial, the secondary benefit of significantly reduced repeat revascularization and the very modest price difference that now exists between drug-eluting stents and bare metal stents means that many interventionalists will continue to use a DES for most patients.
The message from Dr. Pocock and Dr. Stone, underscored by the DANISH and NORSTENT results, is that large and well-run randomized trials can yield important evidence to inform practice that transcends a simple black or white statistical assessment of the primary endpoint.
On Twitter @mitchelzoler
Conventional wisdom holds that for randomized, controlled trials, it’s all about the primary endpoint. If it’s negative or neutral, then none of the other results means much beyond “hypothesis generating.”
This strict-constructionist thinking has now been called into question. A recent article in the New England Journal of Medicine declared “an unreasonable yet widespread practice is the labeling of all randomized trials as either positive or negative on the basis of whether the P value for the primary outcome is less than .05. This view is overly simplistic.” (2016 Sept 1;375[9]:861-70).
The article, by the highly experienced and respected trialists Stuart J. Pocock, PhD, and Gregg W. Stone, MD, adds this: “If the primary outcome is negative, positive findings for secondary outcomes are usually considered to be hypothesis generating. Certainly, regulatory approval of a new drug is unlikely to follow. However, in some instances, secondary findings are compelling enough to affect guidelines and practice.”
This unconventional take from a pair of high-level trialists was especially timely given the buzz around the results from two studies reported at the European Society of Cardiology annual congress in late August, DANISH and NORSTENT.
The DANISH trial compared the impact of implantable cardioverter-defibrillators (ICDs) plus optimal care against optimal care without ICDs in 1,116 patients with nonischemic systolic heart failure. The primary outcome, all-cause death during more than 5 years of follow-up, was a relative 13% less with ICD use, a difference that was not statistically significant, and one secondary outcome, cardiovascular death, was cut by a relative 25% with ICD use, also not statistically significant.
But for the study’s second prespecified secondary endpoint of sudden cardiac death, treatment with ICDs cut the rate in half, compared with nonischemic heart failure patients who did not receive an ICD, a 4-percentage-point difference that was statistically significant.
And in a prespecified secondary analysis of the primary endpoint that broke down the study group by age, the two-thirds of patients younger than 68 years had a significant reduction in all-cause mortality with ICD use, a benefit not seen in patients aged 68 or older.
Discussion of the results at the meeting mainly focused on what meaning, if any, could be drawn from these strongly positive secondary outcomes in a trial neutral for its primary outcome.
“The ICDs did what they were supposed to, prevent sudden cardiac death,” said the lead investigator of the study, Lars Køber, MD. “As a principle I say don’t believe in a subgroup, but guidelines are often based on subgroup analyses.”
“The primary outcome was neutral, but the reduction in sudden cardiac death, the primary objective of an ICD, was significant, so an ICD should be taken into consideration,” commented Michel Komajda, MD, a discussant for the report.
After I wrote a news article about the DANISH report at ESC, I received an email from a reader who objected to spinning the results this way and insisted that no valid lessons can be drawn from the DANISH results because the study’s primary endpoint failed to show a statistical significance. This purist view misses the important, relevant lessons from the DANISH results. The DANISH trial was not designed to provide pivotal data for regulatory approval of ICDs in these patients. Rather, Dr. Køber and his associates designed DANISH to see whether ICD use in these patients could cut all-cause death over a fairly long follow-up. It was a very high bar and ICDs failed, but the deck was stacked against an ICD win. Enrolled patients averaged 64 years old at entry into the study, and they all had New York Heart Association class II or III heart failure. “The overall survival curves start to diverge, but then converge after 5 years because of the comorbidities and patients dying for other reasons,” Dr. Køber noted.
“The message is, in younger patients with less morbidity and more life expectancy, sudden cardiac death is a bigger problem, and they had a substantial drop in mortality” with ICD use, commented heart failure specialist Javed Butler, MD. “It’s very consistent with the way we think about providing ICD treatment to patients.”
In other words, the DANISH results showed that all patients with nonischemic systolic heart failure can’t expect to live substantially longer during extended follow-up if they get an ICD, because the cut in sudden cardiac death the devices provide eventually gets washed out by the many other risks for death these patients face. But younger, relatively healthier patients might very well see their reduced rate of sudden cardiac death translate into an overall mortality benefit even when they are followed for at least 5 years. That’s important information to help an individual patient decide whether to have an ICD placed, and an important message from the DANISH trial despite the neutral primary endpoint.
NORSTENT involved a similar scenario in a trial that addressed a totally different issue: Should patients with either stable or unstable coronary artery disease who are undergoing coronary stenting receive a drug-eluting stent (DES) or a bare metal stent (BMS)? The trial randomized 9,013 patients to receive either of the two stent types plus optimal medical therapy. The primary endpoint was the rate of all-cause death or nonfatal MI during 5 years of follow-up, and the results showed no statistically significant difference between the patients who received a DES and those who got a BMS (N Engl J Med. 2016 Aug 30. doi: 10.1056/NEJMoa1607991).
But for the secondary endpoint of repeat revascularizations performed during follow-up, the use of a DES cut the procedure rate by 3.3 percentage points, a 17% relative risk reduction that was statistically significant. The use of a DES also cut the stent thrombosis rate by 0.4 percentage points, a one-third relative drop in these events that was also statistically significant.
In short, despite the neutral primary endpoint for the trial, the results showed that drug-eluting stents did what they were designed to do relative to bare metal stents: cut the rate of target lesion restenosis and the need for repeat revascularization. Several interventional cardiologists who heard the results at the meeting said that the findings would not change their practice and that they would continue to use the DES as their default device for percutaneous coronary interventions. Although “the long-term benefit of contemporary DES over BMS was less than expected,” said Kaare H. Bønaa, MD, lead investigator for the NORSTENT trial, the secondary benefit of significantly reduced repeat revascularization and the very modest price difference that now exists between drug-eluting stents and bare metal stents means that many interventionalists will continue to use a DES for most patients.
The message from Dr. Pocock and Dr. Stone, underscored by the DANISH and NORSTENT results, is that large and well-run randomized trials can yield important evidence to inform practice that transcends a simple black or white statistical assessment of the primary endpoint.
On Twitter @mitchelzoler
