A large percentage of studies in neurology and psychiatry over the past decade have failed to account for differences between the sexes, according to a team of Canadian researchers.
Dr. Liisa Galea
“Despite the fact there are papers that are using males and females in the studies, they’re not using the males and females in the way that would optimally find the possibility of sex differences,” lead author Liisa A.M. Galea, PhD, told this news organization. Dr. Galea is a professor and distinguished scholar at the Djavad Mowafaghian Center for Brain Health at the University of British Columbia in Vancouver.
The study was published online in Nature Communications.
Optimal design uncommon
Differences in how neurologic and psychiatric diseases affect men and women have been well documented. Women, for example, are more susceptible to severe stroke, and men are more prone to cognitive decline with schizophrenia. With Alzheimer’s disease, women typically have more severe cognitive defects.
The researchers surveyed 3,193 papers that included a multitude of studies. Although most of the papers reported studies that included both sexes, only 19% of surveyed studies used what Dr. Galea called an optimal design for the discovery of sex differences. “What I mean by ‘optimally’ is the design of the experiments and the analysis of sex as a variable,” she said. And in 2019, only 5% of the studies used sex as a variable for determining differences between the sexes, the study found.
In the current research, two authors read the methods and results of each study described in each paper, Dr. Galea said. The readers noted whether the paper reported the study sample size and whether the studies used a balanced design. The surveyed journals include Nature Neuroscience, Neuron, Journal of Neuroscience, Molecular Psychiatry, Biological Psychiatry, and Neuropsychopharmacology.
‘Not much is changing’
“I had a suspicion that this was happening,” Dr. Galea said. “I didn’t know that it’s so bad, to be fair.” The “good news story,” she said, is that more papers considered sex as a factor in the later years surveyed. In 2019, more than 95% of papers across both disciplines reported participants’ sex, compared with about 70% in 2009. However, less than 20% of the papers in all study years reported studies that used sex optimally to determine differences between the sexes.
“The other thing that shocked me,” Dr. Galea said, “was that even despite the fact that we saw this increase in the number of papers that were using males and females, we didn’t see the sort of corresponding increase in those that were using ‘optimal design’ or ‘optimal analysis,’ ” Dr. Galea said. In 2009, 14% of papers used optimal design and 2% used optimal analysis for determining sex differences. By 2019, those percentages were 19% and 5%, respectively.
But even the papers that used both sexes had shortcomings, the study found. Just over one-third of these papers (34.5%) didn’t use a balanced design. Just over one-quarter (25.9%) didn’t identify the sample size, a shortcoming that marked 18% of these studies in 2009 and 33% in 2019. Fifteen percent of papers examined included studies that used both sexes inconsistently.
“That matters, because other studies have found that about 20% of papers are doing some kind of analysis with sex, but we had a suspicion that a lot of studies would include sex as a covariate,” Dr. Galea said. “Essentially what that does is, you remove that variable from the data. So, any statistical variation due to sex is then gone.
“The problem with that,” she added, “is you’re not actually looking to see if there’s an influence of sex; you’re removing it.”
Dr. Galea noted that this study points to a need for funding agencies to demand that researchers meet their mandates on sex- and gender-based analysis. “Despite the mandates, not much is really changing as far as the analysis or design of experiments, and we need to figure out how to change that,” she said. “We need to figure out how to get researchers more interested to use the power of studying sex differences.”