How did you convert the promise of CNN into tangible results?
The plan was simple: Collect endoscopic images from patients with IBD, find some experts to grade IBD severity on the images, and train a CNN model using the images and expert labels.
In 2016, developing a CNN wasn’t easy. There was no database of endoscopic images or simple methods for image labeling. The CNN needed tens of thousands of images. How were we to collect enough images with a broad range of IBD severity? I also reached some technical limits and needed help solving computational challenges.
Designing our first IBD endoscopic CNN took years of reading, coursework, additional training, and a new host of collaborators.
Failure was frequent, and my colleagues and I spent a lot of nights and weekends looking at thousands of individual endoscopic images. But we eventually had a working model for grading endoscopic severity, and its performance exceeded our expectations.
To our surprise, the CNN model grading of ulcerative colitis severity almost perfectly matched the opinion of IBD experts. We introduced the proof of concept that AI could automate complex disease measurement for IBD.
What took us 3 years in 2016 would take about 3 weeks today.
You have said that AI could help reduce the substantial administrative burdens in medicine today. What might an AI-assisted future look like for time-strapped gastroenterologists?
We will be spending more time on complex decision-making and developing treatment plans, with less time needed to hunt for information in the chart and administrative tasks.
The practical applications of AI will chip away at tedious mechanical tasks, soon to be done by machines, reclaiming time for gastroenterologists.
For example, automated documentation is almost usable, and audio recordings in the clinic could be leveraged to generate office notes.
Computer vision analysis of endoscopic video is generating draft procedural notes and letters to patients in a shared language, as well as recommending surveillance intervals based on the findings.
Text processing is already being used to automate billing and manage health maintenance like vaccinations, laboratory screening, and therapeutic drug monitoring.
Unfortunately, I don’t think that AI will immediately help with burnout. These near-term AI administrative assistant advantages, however, will help us manage the increasing patient load, address physician shortages, and potentially improve access to care in underserved areas.
Were there any surprises in your work?
I must admit, I was certain AI would put us gastroenterologists to shame. Over time, I have reversed that view.
AI really struggles to understand the holistic patient context when interpreting disease and predicting what to do for an individual patient. Humans anticipate gaps in data and customize the weighting of information when making decisions for individuals. An experienced gastroenterologist can incorporate risks, harms, and costs in ways AI is several generations from achieving.
With certainty, AI will outperform gastroenterologists for tedious and repetitive tasks, and we should gladly expect AI to assume those responsibilities. However, many unknowns remain in the daily management of GI conditions. We will continue to rely on the clinical experience, creativity, and improvisation of gastroenterologists for years to come.
