An automated artificial intelligence (AI) model trained to read electroencephalograms (EEGs) in patients with suspected epilepsy is just as accurate as trained neurologists, new data suggest.
Known as SCORE-AI, the technology distinguishes between abnormal and normal EEG recordings and classifies irregular recordings into specific categories crucial for patient decision-making.
“SCORE-AI can be used in place of experts in underprivileged areas, where expertise is missing, or to help physicians to preselect or prescore recordings in areas where the workload is high – we can all benefit from AI,” study investigator Sándor Beniczky, MD, PhD, said in a JAMA Neurology podcast.
Dr. Beniczky is professor of clinical neurophysiology at Aarhus University in Denmark.
The findings were published online in JAMA Neurology.
Gaining a foothold
Increasingly, AI is gaining a foothold in medicine by credibly addressing patient queries and aiding radiologists.
To bring AI to EEG interpretation, the researchers developed and validated an AI model that was able to assess routine, clinical EEGs in patients with suspected epilepsy.
Beyond using AI to distinguish abnormal from normal EEG recordings, the researchers wanted to train the new system to classify abnormal recordings into the major categories that are most relevant for clinical decision-making in patients who may have epilepsy. The categories included epileptiform-focal, epileptiform-generalized, nonepileptiform-focal, and nonepileptiform-diffuse abnormalities.
The researchers trained the learning model using Standardized Computer-based Organized Reporting of EEG (SCORE) software.
In the development phase, the model was trained using more than 30,490 anonymized and highly annotated EEG recordings from 14,100 men (median age, 25 years) from a single center. The recordings had an average duration of 31 minutes and were interpreted by 17 neurologists using standardized criteria. If an EEG recording was abnormal, the physicians had to specify which abnormal features were present.
SCORE-AI then performed an analysis of the recordings based on input from the experts.
To validate the findings, investigators used two independent test datasets. The first dataset consisted of 100 representative routine EEGs from 61 men (median age, 26 years), evaluated by 11 neurologists from different centers.
The consensus of these evaluations served as the reference standard. The second dataset comprised nearly 10,000 EEGs from a single center (5,170 men; median age, 35 years), independently assessed by 14 neurologists.
Near-perfect accuracy
When compared with the experts, SCORE-AI had near-perfect accuracy with an area under the receiver operating characteristic (AUROC) curve for differentiating normal from abnormal EEG recordings of 0.95.
SCORE-AI also performed well at identifying generalized epileptiform abnormalities (AUROC, 0.96), focal epileptiform abnormalities (AUROC, 0.91), focal nonepileptiform abnormalities (AUROC, 0.89), and diffuse nonepileptiform abnormalities (AUROC, 0.93).
In addition, SCORE-AI had excellent agreement with clinicians – and sometimes agreed with individual experts more than the experts agreed with one another.
When Dr. Beniczky and team tested SCORE-AI against three previously published AI models, SCORE-AI demonstrated greater specificity than those models (90% vs. 3%-63%) but was not as sensitive (86.7%) as two of the models (96.7% and 100%).
One of the study’s limitations was the fact that SCORE-AI was developed and validated on routine EEGs that excluded neonates and critically ill patients.
In the future, Dr. Beniczky said on the podcast, the team would like to train SCORE-AI to read EEGs with more granularity, and eventually use only one single channel to record EEGs. At present, SCORE-AI is being integrated with Natus Neuro, a widely used EEG equipment system, the investigators note.
In an accompanying editorial, Jonathan Kleen, MD, PhD, and Elan Guterman, MD, said, “The overall approach taken ... in developing and validating SCORE-AI sets a standard for this work going forward.”
Dr. Kleen and Dr. Guterman note that the technological gains brought about by SCORE-AI technology “could offer an exciting prospect to improve EEG availability and clinical care for the 50 million people with epilepsy worldwide.”
A version of this article originally appeared on Medscape.com.