a new study suggests.
In the test dataset, the video-based algorithm, called a deep neural network (DNN), discriminated reduced LVEF (< 40%) with an area under the receiver operating characteristic curve of 0.911.
In the external validation dataset, the DNN discriminated reduced LVEF with an AUROC of 0.906. However, the DNN tended to overestimate low LVEFs and to underestimate high LVEFs.
“We know the findings will be unexpected for cardiologists who don’t typically expect to get an estimate of systolic function or pump function just from an angiogram,” principal investigator Geoffrey H. Tison, MD, of the University of California, San Francisco, said in an interview.
In fact, he noted, “one of the challenges we face is a lack of trust by the health care community. They may not understand what drives the predictions behind our models. We have to translate that information in such a way that physicians trust that the algorithm is using the right features from the data they feed in to make the predictions.”
To help bolster that trust, “we display the ‘Model Facts,’ a nutrition-style label that describes how we train the algorithm, how it was validated, and the inclusion and exclusion criteria,” added lead author Robert Avram, MD, of the University of Montreal.
Model Facts is a safeguard against inappropriate use of the algorithm, Dr. Avram said. For example, if the algorithm was trained on patients between the ages of 40 and 90 and a clinician fed in data for a 35-year-old, a pop-up would appear warning the physician that the data being inputted are different from the data the algorithm was trained and validated on, and so any prediction “should be taken with a grain of salt.”
The study was published online in JAMA Cardiology.
Additional procedure
LVEF can be determined before coronary angiography with transthoracic echocardiography, but that is not always available, particularly for patients being seen emergently for acute coronary syndromes, the researchers wrote. LVEF can also be assessed using left ventriculography, an additional procedure that requires insertion of a pigtail catheter into the left ventricle and injection of more contrast and longer radiation exposure.
“Novel methods to assess LVEF at the point of care during coronary angiography would expand the available options to perform this important physiologic determination,” they wrote. “Video-based deep neural networks can learn subtle patterns from medical data to accomplish certain tasks beyond what physicians can achieve with that data, providing an opportunity to assess cardiac systolic function in real time from standard angiographic images without additional cost or procedures.”
The investigators conducted a cross-sectional study using UCSF patient data from 2012 to 2019. Data were randomly categorized into training, development, and test datasets.
External validation data were obtained from the University of Ottawa Heart Institute.
All adult patients who received a coronary angiogram and a transthoracic echocardiogram (TTE) within 3 months before or 1 month after receiving the angiogram were included.
A total of 4,042 angiograms with corresponding TTE LVEF from 3,679 UCSF patients were included in the analysis. The mean age of the patients was 64.3 years, and 65% were men.
The researchers’ video-based DNN, called CathEF, was used to discriminate reduced LVEF and to predict a continuous LVEF percentage from standard angiogram videos of the left coronary artery.
In the UCSF test dataset, CathEF discriminated reduced LVEF with an AUROC of 0.911; the diagnostic odds ratio for reduced LVEF was 22.7.
Furthermore, the CathEF-predicted that LVEF had a mean absolute error (MAE) of 8.5%, compared with TTE LVEF.
The CathEF-predicted LVEF differed 5% or less in comparison with the TTE LVEF in 38% of the test dataset studies; however, differences greater than 15% were seen in 15.2% of cases.
In the external validation, CathEF discriminated reduced LVEF with an AUROC of 0.906 and an MAE of 7%.
CathEF performance was consistent irrespective of patient characteristics, including sex, body mass index, low estimated glomerular filtration rate (< 45), acute coronary syndromes, obstructive coronary artery disease, and left ventricular hypertrophy.
However, as noted, it tended to overestimate low LVEFs and to underestimate high LVEFs.
“Further research can improve accuracy and reduce the variability of DNNs to maximize their clinical utility,” the authors concluded.
A validation study is underway at the Montreal Heart Institute, and similar studies are planned at UCSF and McGill University, Dr. Tison said. “We expect to present preliminary findings at medical conferences either before the end of the year or maybe for the American College of Cardiology meeting in March 2024.”