ORLANDO — Experts shed light on the applications, benefits, and pitfalls of artificial intelligence (AI) during the Merrit-Putnam Symposium at the annual meeting of the American Epilepsy Society (AES).
In a session titled “Artificial Intelligence Fundamentals and Breakthrough Applications in Epilepsy,” University of Pittsburgh neurologist and assistant professor Wesley Kerr, MD, PhD, provided an overview of AI as well its applications in neurology. He began by addressing perhaps one of the most controversial topics regarding AI in the medical community: clinicians’ fear of being replaced by technology.
he told the audience.
To Optimize AI, Clinicians Must Lay the Proper Foundation
Dr. Kerr’s presentation focused on providing audience members with tools to help them evaluate new technologies, recognize benefits, and identify key costs and limitations associated with AI implementation and integration into clinical practice.
Before delving deeper, one must first understand basic terminology regarding AI. Without this knowledge, clinicians may inadvertently introduce bias or errata or fail to understand how to best leverage the technology to enhance the quality of the practice while improving patient outcomes.
Machine learning (ML) describes the process of using data to learn a specific task. Deep learning (DL) stacks multiple layers of ML to improve performance on the task. Lastly, generative AI generates content such as text, images, and media.
Utilizing AI effectively in clinical applications involves tapping into select features most related to prediction (for example, disease factors) and grouping features into categories based on measuring commonalities such as factor composition in a population. This information should be used in training data only.
Fully understanding ML/AI allows clinicians to use it as a diagnostic test by exploiting a combination of accuracy, sensitivity, and specificity, along with positive and negative predictive values.
Data Fidelity and Integrity Hinge on Optimal Data Inputs
In the case of epilepsy, calibration curves can provide practical guidance in terms of predicting impending seizures.
“ML/AI needs gold-standard labels for evaluation,” Dr. Kerr said. He went on to stress the importance of quality data inputs to optimize the fidelity of AI’s predictive analytics.
“If you input garbage, you’ll get garbage out,” he said. “So a lot of garbage going in means a lot of garbage out.”
Such “garbage” can result in missed or erroneous diagnoses, or even faulty predictions. Even when the data are complete, AI can draw incorrect conclusions based on trends for which it lacks proper context.
Dr. Kerr used epilepsy trends in the Black population to illustrate this problem.
“One potential bias is that AI can figure out a patient is Black without being told, and based on data that Black patients are less likely to get epilepsy surgery,” he said, “AI would say they don’t need it because they’re Black, which isn’t true.”
In other words, ML/AI can use systematic determinants of health, such as race, to learn what Dr. Kerr referred to as an “inappropriate association.”
For that reason, ML/AI users must test for bias.
Such data are often retrieved from electronic health records (EHR), which serve as an important source of data ML/AI input. Using EHR makes sense, as they are a major source of missed potential in improving prompt treatment. According to Dr. Kerr, 20% of academic neurologists’ notes miss seizure frequency, and 30% miss the age of onset.
In addition, International Classification of Diseases (ICD) codes create another hurdle depending on the type of code used. For example, epilepsy with G40 or 2 codes of R56 is reliable, while focal to bilateral versus generalized epilepsy proves more challenging.