FDA’s position
Sameer Jauhar, MBChB, PhD, a consultant psychiatrist at the South London and Maudsley NHS Foundation Trust, London, was equally unimpressed with the quality of the review.
“I try to approach everything with an open mind,” he said. “But as a doctor, if I’m reading about evidence, I expect a well-conducted meta-analysis and/or very clear systematic review with an adequate level of peer review.”
The current review, said Jauhar, doesn’t meet these criteria. He said the article reads more like a narrative review, one in which the authors dictated their own arbitrary criteria of study quality.
Most important, he said, is the fact that ECT has been the focus of a great deal of research. “The best quality synthesis of the evidence I’ve come across is the UK ECT Review Group’s 2003 meta-analysis, published in The Lancet, which asked this very question. None of the studies has changed since then.”
Jauhar also noted that in 2018, the Food and Drug Administration reclassified ECT from class III (higher risk) to class II (moderate risk). Use of ECT was also limited to treatment of “catatonia or a severe major depressive episode associated with major depressive disorder or bipolar disorder in patients age 13 years and older who are treatment-resistant or who require a rapid response treatment due to the severity of their psychiatric or medical condition.”
The FDA also noted that “[t]he safe use of ECT for treatment of these conditions has been well studied and is better understood than other uses. Therefore, sufficient information exists to establish special controls that mitigate known risks and provide a reasonable assurance of safety and effectiveness for these two uses of ECT devices.”
As part of the FDA’s 2018 reclassification, ECT manufacturers were required to file a premarket approval application for all uses that were not reclassified. The full text of the order is available in the Federal Register.
“So the FDA has been through it, The Lancet has been through it, and the findings were clear,” said Jauhar. “It’s very easy to poke holes in studies that were conducted 30 or more years ago. But the fact is, the field has moved on.”
Jauhar acknowledged there are patients who have had bad experiences with ECT, and he accepts that such events occasionally occur. Nevertheless, he said,
Jauhar also noted that the journal in which the review was published has a 2019 CiteScore of 0.3 and ranks in the 15th percentile of the Scopus Clinical Psychology category. He further noted that Kirsch is a member of the journal’s editorial board.
“I would say that the level of peer review here was negligible at best,” Jauhar said. “In addition, the ‘findings’ are driven more by ideology than evidence.”
Asked to respond to Jauhar’s comments, Kirsch noted that although he is on the journal’s advisory board, he has not been actively involved. Kirsch added that he serves on about a dozen academic advisory boards and serves as a reviewer for many top scientific and medical journals, including The Journal of the American Medical Association and the New England Journal of Medicine.
“Our article was peer reviewed,” Kirsch said, “and we revised it following the first round of reviews.” The review process did not differ from those he has gone through with more than 250 published peer-reviewed articles on which he was an author or coauthor, he noted.