User login
MILAN – Two independent efforts to use artificial intelligence (AI) to predict the development of early rheumatoid arthritis (RA) from patients with signs and symptoms not meeting full disease criteria showed good, near expert-level accuracy, according to findings from two studies presented at the annual European Congress of Rheumatology.
In one study, researchers from Leiden University Medical Center in the Netherlands developed an AI-based method to automatically analyze MR scans of extremities in order to predict early rheumatoid arthritis. The second study involved a Japanese research team that used machine learning to create a model capable of predicting progression from undifferentiated arthritis (UA) to RA. Both approaches would facilitate early diagnosis of RA, enabling timely treatment and improved clinical outcomes.
Lennart Jans, MD, PhD, who was not involved in either study but works with AI-assisted imaging analysis on a daily basis as head of clinics in musculoskeletal radiology at Ghent University Hospital and a professor of radiology at Ghent University in Belgium, said that integrating AI into health care poses several challenging aspects that need to be addressed. “There are three main challenges associated with the development and implementation of AI-based tools in clinical practice,” he said. “Firstly, obtaining heterogeneous datasets from different image hardware vendors, diverse racial and ethnic backgrounds, and various ages and genders is crucial for training and testing the AI algorithms. Secondly, AI algorithms need to achieve a predetermined performance level depending on the specific use case. Finally, a regulatory pathway must be followed to obtain the necessary FDA or MDR [medical devices regulation] certification before applying an AI use case in clinical practice.”
RA prediction
Yanli Li, the first author of the study and a member of the division of image processing at Leiden University Medical Center, explained the potential benefits of early RA prediction. “If we could determine whether a patient presenting with clinically suspected arthralgia (CSA) or early onset arthritis (EAC) is likely to develop RA in the near future, physicians could initiate treatment earlier, reducing the risk of disease progression.”
Currently, rheumatologists estimate the likelihood of developing RA by visually scoring MR scans using the RAMRIS scoring system. “We decided to explore the use of AI,” Dr. Li explained, “because it could save time, reduce costs and labor, eliminate the need for scoring training, and allow for hypothesis-free discoveries.”
The research team collected MR scans of the hands and feet from Leiden University Medical Center’s radiology department. The dataset consisted of images from 177 healthy individuals, 692 subjects with CSA (including 113 who developed RA), and 969 with EAC (including 447 who developed RA). The images underwent automated preprocessing to remove artifacts and standardize the input for the computer. Subsequently, a deep learning model was trained to predict RA development within a 2-year time frame.
The training process involved several steps. Initially, the researchers pretrained the model to learn anatomy by masking parts of the images and tasking the computer with reconstructing them. Subsequently, the AI was trained to differentiate between the groups (EAC vs. healthy and CSA vs. healthy), then between RA and other disorders. Finally, the AI model was trained to predict RA.
The accuracy of the model was evaluated using the area under the receiver operator characteristic curve (AUROC). The model that was trained using MR scans of the hands (including the wrist and metacarpophalangeal joints) achieved a mean AUROC of 0.84 for distinguishing EAC from healthy subjects and 0.83 for distinguishing CSA from healthy subjects. The model trained using MR scans of both the hands and feet achieved a mean AUROC of 0.71 for distinguishing RA from non-RA cases in EAC. The accuracy of the model in predicting RA using MR scans of the hands was 0.73, which closely matches the reported accuracy of visual scoring by human experts (0.74). Importantly, the generation and analysis of heat maps suggested that the deep learning model predicts RA based on known inflammatory signals.
“Automatic RA prediction using AI interpretation of MR scans is feasible,” Dr. Li said. “Incorporating additional clinical data will likely further enhance the AI prediction, and the heat maps may contribute to the discovery of new MRI biomarkers for RA development.”
“AI models and engines have achieved near-expertise levels for various use cases, including the early detection of RA on MRI scans of the hands,” said Dr. Jans, the Ghent University radiologist. “We are observing the same progress in AI detection of rheumatic diseases in other imaging modalities, such as radiography, CT, and ultrasound. However, it is important to note that the reported performances often apply to selected cohorts with standardized imaging protocols. The next challenge [for Dr. Li and colleagues, and others] will be to train and test these algorithms using more heterogeneous datasets to make them applicable in real-world settings.”
A ‘transitional phase’ of applying AI techniques
“In a medical setting, as computer scientists, we face unique challenges,” pointed out Berend C. Stoel, MSc, PhD, the senior author of the Leiden study. “Our team consists of approximately 30-35 researchers, primarily electrical engineers or computer scientists, situated within the radiology department of Leiden University Medical Center. Our focus is on image processing, seeking AI-based solutions for image analysis, particularly utilizing deep learning techniques.”
Their objective is to validate this method more broadly, and to achieve that, they require collaboration with other hospitals. Up until now, they have primarily worked with a specific type of MR images, extremity MR scans. These scans are conducted in only a few centers equipped with extremity MR scanners, which can accommodate only hands or feet.
“We are currently in a transitional phase, aiming to apply our methods to standard MR scans, which are more widely available,” Dr. Stoel informed this news organization. “We are engaged in various projects. One project, nearing completion, involves the scoring of early RA, where we train the computer to imitate the actions of rheumatologists or radiologists. We started with a relatively straightforward approach, but AI offers a multitude of possibilities. In the project presented at EULAR, we manipulated the images in a different manner, attempting to predict future events. We also have a parallel project where we employ AI to detect inflammatory changes over time by analyzing sequences of images (MR scans). Furthermore, we have developed AI models to distinguish between treatment and placebo groups. Once the neural network has been trained for this task, we can inquire about the location and timing of changes, thereby gaining insights into the therapy’s response.
“When considering the history of AI, it has experienced both ups and downs. We are currently in a promising phase, but if certain projects fail, expectations might diminish. My hope is that we will indeed revolutionize and enhance disease diagnosis, monitoring, and prediction. Additionally, AI may provide us with additional information that we, as humans, may not be able to extract from these images. However, it is difficult to predict where we will stand in 5-10 years,” he concluded.
Predicting disease progression
The second study, which explored the application of AI in predicting the progression of undifferentiated arthritis (UA) to RA, was presented by Takayuki Fujii, MD, PhD, assistant professor in the department of advanced medicine for rheumatic diseases at Kyoto University’s Graduate School of Medicine in Japan. “Predicting the progression of RA from UA remains an unmet medical need,” he reminded the audience.
Dr. Fujii’s team used data from the KURAMA cohort, a large observational RA cohort from a single center, to develop a machine learning model. The study included a total of 322 patients initially diagnosed with UA. The deep neural network (DNN) model was trained using 24 clinical features that are easily obtainable in routine clinical practice, such as age, sex, C-reactive protein (CRP) levels, and disease activity score in 28 joints using erythrocyte sedimentation rate (DAS28-ESR). The DNN model achieved a prediction accuracy of 85.1% in the training cohort. When the model was applied to validation data from an external dataset consisting of 88 patients from the ANSWER cohort, a large multicenter observational RA cohort, the prediction accuracy was 80%.
“We have developed a machine learning model that can predict the progression of RA from UA using clinical parameters,” Dr. Fujii concluded. “This model has the potential to assist rheumatologists in providing appropriate care and timely intervention for patients with UA.”
“Dr. Fujii presented a fascinating study,” Dr. Jans said. “They achieved an accuracy of 80% when applying a DNN model to predict progression from UA to RA. This level of accuracy is relatively high and certainly promising. However, it is important to consider that a pre-test probability of 30% [for progressing from UA to RA] is also relatively high, which partially explains the high accuracy. Nonetheless, this study represents a significant step forward in the clinical management of patients with UA, as it helps identify those who may benefit the most from regular clinical follow-up.”
Dr. Li and Dr. Stoel report no relevant financial relationships with industry. Dr. Fujii has received speaking fees from Asahi Kasei, AbbVie, Chugai, and Tanabe Mitsubishi Pharma. Dr. Jans has received speaking fees from AbbVie, UCB, Lilly, and Novartis; he is cofounder of RheumaFinder. The Leiden study was funded by the Dutch Research Council and the China Scholarship Council. The study by Dr. Fujii and colleagues had no outside funding.
A version of this article first appeared on Medscape.com.
MILAN – Two independent efforts to use artificial intelligence (AI) to predict the development of early rheumatoid arthritis (RA) from patients with signs and symptoms not meeting full disease criteria showed good, near expert-level accuracy, according to findings from two studies presented at the annual European Congress of Rheumatology.
In one study, researchers from Leiden University Medical Center in the Netherlands developed an AI-based method to automatically analyze MR scans of extremities in order to predict early rheumatoid arthritis. The second study involved a Japanese research team that used machine learning to create a model capable of predicting progression from undifferentiated arthritis (UA) to RA. Both approaches would facilitate early diagnosis of RA, enabling timely treatment and improved clinical outcomes.
Lennart Jans, MD, PhD, who was not involved in either study but works with AI-assisted imaging analysis on a daily basis as head of clinics in musculoskeletal radiology at Ghent University Hospital and a professor of radiology at Ghent University in Belgium, said that integrating AI into health care poses several challenging aspects that need to be addressed. “There are three main challenges associated with the development and implementation of AI-based tools in clinical practice,” he said. “Firstly, obtaining heterogeneous datasets from different image hardware vendors, diverse racial and ethnic backgrounds, and various ages and genders is crucial for training and testing the AI algorithms. Secondly, AI algorithms need to achieve a predetermined performance level depending on the specific use case. Finally, a regulatory pathway must be followed to obtain the necessary FDA or MDR [medical devices regulation] certification before applying an AI use case in clinical practice.”
RA prediction
Yanli Li, the first author of the study and a member of the division of image processing at Leiden University Medical Center, explained the potential benefits of early RA prediction. “If we could determine whether a patient presenting with clinically suspected arthralgia (CSA) or early onset arthritis (EAC) is likely to develop RA in the near future, physicians could initiate treatment earlier, reducing the risk of disease progression.”
Currently, rheumatologists estimate the likelihood of developing RA by visually scoring MR scans using the RAMRIS scoring system. “We decided to explore the use of AI,” Dr. Li explained, “because it could save time, reduce costs and labor, eliminate the need for scoring training, and allow for hypothesis-free discoveries.”
The research team collected MR scans of the hands and feet from Leiden University Medical Center’s radiology department. The dataset consisted of images from 177 healthy individuals, 692 subjects with CSA (including 113 who developed RA), and 969 with EAC (including 447 who developed RA). The images underwent automated preprocessing to remove artifacts and standardize the input for the computer. Subsequently, a deep learning model was trained to predict RA development within a 2-year time frame.
The training process involved several steps. Initially, the researchers pretrained the model to learn anatomy by masking parts of the images and tasking the computer with reconstructing them. Subsequently, the AI was trained to differentiate between the groups (EAC vs. healthy and CSA vs. healthy), then between RA and other disorders. Finally, the AI model was trained to predict RA.
The accuracy of the model was evaluated using the area under the receiver operator characteristic curve (AUROC). The model that was trained using MR scans of the hands (including the wrist and metacarpophalangeal joints) achieved a mean AUROC of 0.84 for distinguishing EAC from healthy subjects and 0.83 for distinguishing CSA from healthy subjects. The model trained using MR scans of both the hands and feet achieved a mean AUROC of 0.71 for distinguishing RA from non-RA cases in EAC. The accuracy of the model in predicting RA using MR scans of the hands was 0.73, which closely matches the reported accuracy of visual scoring by human experts (0.74). Importantly, the generation and analysis of heat maps suggested that the deep learning model predicts RA based on known inflammatory signals.
“Automatic RA prediction using AI interpretation of MR scans is feasible,” Dr. Li said. “Incorporating additional clinical data will likely further enhance the AI prediction, and the heat maps may contribute to the discovery of new MRI biomarkers for RA development.”
“AI models and engines have achieved near-expertise levels for various use cases, including the early detection of RA on MRI scans of the hands,” said Dr. Jans, the Ghent University radiologist. “We are observing the same progress in AI detection of rheumatic diseases in other imaging modalities, such as radiography, CT, and ultrasound. However, it is important to note that the reported performances often apply to selected cohorts with standardized imaging protocols. The next challenge [for Dr. Li and colleagues, and others] will be to train and test these algorithms using more heterogeneous datasets to make them applicable in real-world settings.”
A ‘transitional phase’ of applying AI techniques
“In a medical setting, as computer scientists, we face unique challenges,” pointed out Berend C. Stoel, MSc, PhD, the senior author of the Leiden study. “Our team consists of approximately 30-35 researchers, primarily electrical engineers or computer scientists, situated within the radiology department of Leiden University Medical Center. Our focus is on image processing, seeking AI-based solutions for image analysis, particularly utilizing deep learning techniques.”
Their objective is to validate this method more broadly, and to achieve that, they require collaboration with other hospitals. Up until now, they have primarily worked with a specific type of MR images, extremity MR scans. These scans are conducted in only a few centers equipped with extremity MR scanners, which can accommodate only hands or feet.
“We are currently in a transitional phase, aiming to apply our methods to standard MR scans, which are more widely available,” Dr. Stoel informed this news organization. “We are engaged in various projects. One project, nearing completion, involves the scoring of early RA, where we train the computer to imitate the actions of rheumatologists or radiologists. We started with a relatively straightforward approach, but AI offers a multitude of possibilities. In the project presented at EULAR, we manipulated the images in a different manner, attempting to predict future events. We also have a parallel project where we employ AI to detect inflammatory changes over time by analyzing sequences of images (MR scans). Furthermore, we have developed AI models to distinguish between treatment and placebo groups. Once the neural network has been trained for this task, we can inquire about the location and timing of changes, thereby gaining insights into the therapy’s response.
“When considering the history of AI, it has experienced both ups and downs. We are currently in a promising phase, but if certain projects fail, expectations might diminish. My hope is that we will indeed revolutionize and enhance disease diagnosis, monitoring, and prediction. Additionally, AI may provide us with additional information that we, as humans, may not be able to extract from these images. However, it is difficult to predict where we will stand in 5-10 years,” he concluded.
Predicting disease progression
The second study, which explored the application of AI in predicting the progression of undifferentiated arthritis (UA) to RA, was presented by Takayuki Fujii, MD, PhD, assistant professor in the department of advanced medicine for rheumatic diseases at Kyoto University’s Graduate School of Medicine in Japan. “Predicting the progression of RA from UA remains an unmet medical need,” he reminded the audience.
Dr. Fujii’s team used data from the KURAMA cohort, a large observational RA cohort from a single center, to develop a machine learning model. The study included a total of 322 patients initially diagnosed with UA. The deep neural network (DNN) model was trained using 24 clinical features that are easily obtainable in routine clinical practice, such as age, sex, C-reactive protein (CRP) levels, and disease activity score in 28 joints using erythrocyte sedimentation rate (DAS28-ESR). The DNN model achieved a prediction accuracy of 85.1% in the training cohort. When the model was applied to validation data from an external dataset consisting of 88 patients from the ANSWER cohort, a large multicenter observational RA cohort, the prediction accuracy was 80%.
“We have developed a machine learning model that can predict the progression of RA from UA using clinical parameters,” Dr. Fujii concluded. “This model has the potential to assist rheumatologists in providing appropriate care and timely intervention for patients with UA.”
“Dr. Fujii presented a fascinating study,” Dr. Jans said. “They achieved an accuracy of 80% when applying a DNN model to predict progression from UA to RA. This level of accuracy is relatively high and certainly promising. However, it is important to consider that a pre-test probability of 30% [for progressing from UA to RA] is also relatively high, which partially explains the high accuracy. Nonetheless, this study represents a significant step forward in the clinical management of patients with UA, as it helps identify those who may benefit the most from regular clinical follow-up.”
Dr. Li and Dr. Stoel report no relevant financial relationships with industry. Dr. Fujii has received speaking fees from Asahi Kasei, AbbVie, Chugai, and Tanabe Mitsubishi Pharma. Dr. Jans has received speaking fees from AbbVie, UCB, Lilly, and Novartis; he is cofounder of RheumaFinder. The Leiden study was funded by the Dutch Research Council and the China Scholarship Council. The study by Dr. Fujii and colleagues had no outside funding.
A version of this article first appeared on Medscape.com.
MILAN – Two independent efforts to use artificial intelligence (AI) to predict the development of early rheumatoid arthritis (RA) from patients with signs and symptoms not meeting full disease criteria showed good, near expert-level accuracy, according to findings from two studies presented at the annual European Congress of Rheumatology.
In one study, researchers from Leiden University Medical Center in the Netherlands developed an AI-based method to automatically analyze MR scans of extremities in order to predict early rheumatoid arthritis. The second study involved a Japanese research team that used machine learning to create a model capable of predicting progression from undifferentiated arthritis (UA) to RA. Both approaches would facilitate early diagnosis of RA, enabling timely treatment and improved clinical outcomes.
Lennart Jans, MD, PhD, who was not involved in either study but works with AI-assisted imaging analysis on a daily basis as head of clinics in musculoskeletal radiology at Ghent University Hospital and a professor of radiology at Ghent University in Belgium, said that integrating AI into health care poses several challenging aspects that need to be addressed. “There are three main challenges associated with the development and implementation of AI-based tools in clinical practice,” he said. “Firstly, obtaining heterogeneous datasets from different image hardware vendors, diverse racial and ethnic backgrounds, and various ages and genders is crucial for training and testing the AI algorithms. Secondly, AI algorithms need to achieve a predetermined performance level depending on the specific use case. Finally, a regulatory pathway must be followed to obtain the necessary FDA or MDR [medical devices regulation] certification before applying an AI use case in clinical practice.”
RA prediction
Yanli Li, the first author of the study and a member of the division of image processing at Leiden University Medical Center, explained the potential benefits of early RA prediction. “If we could determine whether a patient presenting with clinically suspected arthralgia (CSA) or early onset arthritis (EAC) is likely to develop RA in the near future, physicians could initiate treatment earlier, reducing the risk of disease progression.”
Currently, rheumatologists estimate the likelihood of developing RA by visually scoring MR scans using the RAMRIS scoring system. “We decided to explore the use of AI,” Dr. Li explained, “because it could save time, reduce costs and labor, eliminate the need for scoring training, and allow for hypothesis-free discoveries.”
The research team collected MR scans of the hands and feet from Leiden University Medical Center’s radiology department. The dataset consisted of images from 177 healthy individuals, 692 subjects with CSA (including 113 who developed RA), and 969 with EAC (including 447 who developed RA). The images underwent automated preprocessing to remove artifacts and standardize the input for the computer. Subsequently, a deep learning model was trained to predict RA development within a 2-year time frame.
The training process involved several steps. Initially, the researchers pretrained the model to learn anatomy by masking parts of the images and tasking the computer with reconstructing them. Subsequently, the AI was trained to differentiate between the groups (EAC vs. healthy and CSA vs. healthy), then between RA and other disorders. Finally, the AI model was trained to predict RA.
The accuracy of the model was evaluated using the area under the receiver operator characteristic curve (AUROC). The model that was trained using MR scans of the hands (including the wrist and metacarpophalangeal joints) achieved a mean AUROC of 0.84 for distinguishing EAC from healthy subjects and 0.83 for distinguishing CSA from healthy subjects. The model trained using MR scans of both the hands and feet achieved a mean AUROC of 0.71 for distinguishing RA from non-RA cases in EAC. The accuracy of the model in predicting RA using MR scans of the hands was 0.73, which closely matches the reported accuracy of visual scoring by human experts (0.74). Importantly, the generation and analysis of heat maps suggested that the deep learning model predicts RA based on known inflammatory signals.
“Automatic RA prediction using AI interpretation of MR scans is feasible,” Dr. Li said. “Incorporating additional clinical data will likely further enhance the AI prediction, and the heat maps may contribute to the discovery of new MRI biomarkers for RA development.”
“AI models and engines have achieved near-expertise levels for various use cases, including the early detection of RA on MRI scans of the hands,” said Dr. Jans, the Ghent University radiologist. “We are observing the same progress in AI detection of rheumatic diseases in other imaging modalities, such as radiography, CT, and ultrasound. However, it is important to note that the reported performances often apply to selected cohorts with standardized imaging protocols. The next challenge [for Dr. Li and colleagues, and others] will be to train and test these algorithms using more heterogeneous datasets to make them applicable in real-world settings.”
A ‘transitional phase’ of applying AI techniques
“In a medical setting, as computer scientists, we face unique challenges,” pointed out Berend C. Stoel, MSc, PhD, the senior author of the Leiden study. “Our team consists of approximately 30-35 researchers, primarily electrical engineers or computer scientists, situated within the radiology department of Leiden University Medical Center. Our focus is on image processing, seeking AI-based solutions for image analysis, particularly utilizing deep learning techniques.”
Their objective is to validate this method more broadly, and to achieve that, they require collaboration with other hospitals. Up until now, they have primarily worked with a specific type of MR images, extremity MR scans. These scans are conducted in only a few centers equipped with extremity MR scanners, which can accommodate only hands or feet.
“We are currently in a transitional phase, aiming to apply our methods to standard MR scans, which are more widely available,” Dr. Stoel informed this news organization. “We are engaged in various projects. One project, nearing completion, involves the scoring of early RA, where we train the computer to imitate the actions of rheumatologists or radiologists. We started with a relatively straightforward approach, but AI offers a multitude of possibilities. In the project presented at EULAR, we manipulated the images in a different manner, attempting to predict future events. We also have a parallel project where we employ AI to detect inflammatory changes over time by analyzing sequences of images (MR scans). Furthermore, we have developed AI models to distinguish between treatment and placebo groups. Once the neural network has been trained for this task, we can inquire about the location and timing of changes, thereby gaining insights into the therapy’s response.
“When considering the history of AI, it has experienced both ups and downs. We are currently in a promising phase, but if certain projects fail, expectations might diminish. My hope is that we will indeed revolutionize and enhance disease diagnosis, monitoring, and prediction. Additionally, AI may provide us with additional information that we, as humans, may not be able to extract from these images. However, it is difficult to predict where we will stand in 5-10 years,” he concluded.
Predicting disease progression
The second study, which explored the application of AI in predicting the progression of undifferentiated arthritis (UA) to RA, was presented by Takayuki Fujii, MD, PhD, assistant professor in the department of advanced medicine for rheumatic diseases at Kyoto University’s Graduate School of Medicine in Japan. “Predicting the progression of RA from UA remains an unmet medical need,” he reminded the audience.
Dr. Fujii’s team used data from the KURAMA cohort, a large observational RA cohort from a single center, to develop a machine learning model. The study included a total of 322 patients initially diagnosed with UA. The deep neural network (DNN) model was trained using 24 clinical features that are easily obtainable in routine clinical practice, such as age, sex, C-reactive protein (CRP) levels, and disease activity score in 28 joints using erythrocyte sedimentation rate (DAS28-ESR). The DNN model achieved a prediction accuracy of 85.1% in the training cohort. When the model was applied to validation data from an external dataset consisting of 88 patients from the ANSWER cohort, a large multicenter observational RA cohort, the prediction accuracy was 80%.
“We have developed a machine learning model that can predict the progression of RA from UA using clinical parameters,” Dr. Fujii concluded. “This model has the potential to assist rheumatologists in providing appropriate care and timely intervention for patients with UA.”
“Dr. Fujii presented a fascinating study,” Dr. Jans said. “They achieved an accuracy of 80% when applying a DNN model to predict progression from UA to RA. This level of accuracy is relatively high and certainly promising. However, it is important to consider that a pre-test probability of 30% [for progressing from UA to RA] is also relatively high, which partially explains the high accuracy. Nonetheless, this study represents a significant step forward in the clinical management of patients with UA, as it helps identify those who may benefit the most from regular clinical follow-up.”
Dr. Li and Dr. Stoel report no relevant financial relationships with industry. Dr. Fujii has received speaking fees from Asahi Kasei, AbbVie, Chugai, and Tanabe Mitsubishi Pharma. Dr. Jans has received speaking fees from AbbVie, UCB, Lilly, and Novartis; he is cofounder of RheumaFinder. The Leiden study was funded by the Dutch Research Council and the China Scholarship Council. The study by Dr. Fujii and colleagues had no outside funding.
A version of this article first appeared on Medscape.com.
AT EULAR 2023