User login
.
So far, the U.S. Food and Drug Administration has cleared two devices. Both are computer-aided skin lesion classification devices meant to help clinicians assess cases of suspected melanoma.
Both were given a class III designation. That classification is intended for products that are considered to have a high risk of harm because of flawed design or implementation. Many such devices are under development, and there has been a proposal to include these devices in class II, which is less restrictive.
The FDA turned to one of its expert panels for advice. At a meeting held on Aug. 29, experts on the panel offered differing views and expressed concerns about the accuracy of these devices.
This was the second day of meetings of the general and plastic surgery devices panel of the FDA’s Medical Devices Advisory Committee. On the previous day, the panel held a wide-ranging discussion about expanding use of skin lesion analyzer devices.
The FDA sought the expert panel’s advice concerning a field that appears to be heating up quickly after relatively quiet times.
Two devices have been approved by the FDA so far, but only one is still being promoted – SciBase AB’s Nevisense. The Swedish company announced in May 2020 that it had received FDA approval for Nevisense 3.0, the third generation of their Nevisense system for early melanoma detection, an AI-based point-of-care system for the noninvasive evaluation of irregular moles.
The other device, known as MelaFind, was acquired by Strata Skin Sciences, but the company said in 2017 that it discontinued research and development, sales, and support activity related to the device, according to a filing with the Securities and Exchange Commission.
But there’s been a swell in recent years in the number of publications related to the use of AI and machine learning, which could give rise to new tools for aiding in the diagnosis of skin conditions, including cancer. Google is among the companies that are involved in these efforts.
So, the FDA asked the expert panel to discuss a series of questions related to how the agency should weigh the risks of computer-aided devices for melanoma diagnosis. The agency also asked the panel to provide feedback about how well risks associated with such devices and tools might be managed and to offer suggestions.
The discussion at the July 29 meeting spun beyond narrow questions about reclassification of the current class III devices to topics involving emerging technology, such as efforts to apply AI to dermatology.
“Innovation continues. Medical device developers are anxious to plan how they might be able to develop the level of evidence that would meet your expectations” for future products, Binita Ashar, MD, a senior official in FDA’s Center for Devices and Radiological Health, told the panel.
Company CEO backs tougher regulation
Simon Grant, the chief executive of SciBase, which markets Nevisense, the first and only skin cancer–detecting device currently on the U.S. market, sought to make a case for sticking with the tougher class III regulations.
Speaking during the public comment session, Mr. Grant said switching to class II designations would weaken the standards used in clearing products that analyze skin lesions so as to put patients at risk.
Under the FDA’s rules, the agency designates as class III devices that present potential unreasonable risk of illness or injury. Only about 10% of devices fall into this category. Such devices include implantable pacemakers and breast implants, as well as SciBase’s Nevisense.
About 43% of medical devices fall into the class II category, which includes powered wheelchairs and some pregnancy test kits, the FDA website says.
Class I medical devices pose minimal potential for harm and tend to be simpler in design. These include enema kits and elastic bandages, the FDA says.
Mr. Grant told the meeting that in his career he has worked on two class III products and about 20 class II products. (He had previously worked at medical startups Synectics Medical and Neoventa, as well as established multinationals such as Medtronic.)
“I can tell you that – practically – the FDA has many fewer sticks and much less control when it comes to class II devices,” he said. He offered an example of a manufacturer of a class II device having more latitude in making small changes to products without notifying the FDA.
In his hypothetical example, such a change could have unintended consequences, and “with AI systems, small changes can result in large and nonlinear or even random effects,” Mr. Grant said. “But it’s too late if the product is on the market and the harm has already occurred,” he said.
The American Society for Dermatologic Surgery Association also protested the reclassifying of approved computer-aided melanoma detection class III devices.
In a statement posted on the FDA website as part of the materials for the meeting, the ASDSA raised a series of concerns about the prospects of expanded U.S. use of tools for assisting in diagnosing melanoma, including ones that would be marketed to consumers.
“To the extent that algorithms and devices for patient self-diagnosis of skin lesions are already widely available, they should be required to include detailed disclaimers that include that they are for entertainment and educational purposes and not a diagnostic device, that they are not approved by dermatologists or a recognized medical regulatory authority for self-diagnosis,” the ASDSA said.
Devices and algorithms in screening tools “are not highly regulated and remain unproven. They may result in wrong diagnoses, missed diagnoses, or over- or underdiagnosis,” the ASDSA added. “Both patients at low risk and those at high risk are better served by scheduling an in-person examination with a board-certified dermatologist, who can also help them determine the appropriate future skin screening schedule that is most appropriate for them.”
‘Stepping stone’
However, there is strong consumer demand for better information about skin conditions, and many patients face hurdles in going to dermatologists.
Google research has shown that consumers are seeking “a stepping stone” between the information they can easily find online and what they could get from a medical professional, said Lily Peng, MD, PhD, a director of product management for the health AI team at Google. Dr. Peng was a scheduled presenter at the July 29 meeting.
Consumers often are looking for more information on common conditions such as acne and poison ivy, and they sometimes face challenges in getting access to clinicians, she said.
“There are many unmet needs for consumers experiencing skin issues, many of which are lower-acuity conditions. There’s a big opportunity to increase accessibility and relevance of health journeys for consumers,” Dr. Peng said. “We have heard from consumers that they would like to have a self-help tool for nonserious conditions so they can decide when to seek medical attention.”
Dr. Peng’s presentation was not directly related to the question of class II or class III designation for existing products. Instead, her talk served as a glimpse into the work already underway in creating apps and tools for consumers.
Google researchers have published a number of studies in recent years about the use of AI to improve dermatology diagnosis.
A 2020 article reported on Google’s test of a form of AI known as deep learning system (DLS) to provide a differential diagnosis of skin conditions. On 963 validation cases, where a rotating panel of three board-certified dermatologists defined the reference standard, the DLS was noninferior to six other dermatologists and was superior to six primary care physicians (PCPs) and six nurse practitioners (NPs), according to a summary of the article.
A 2021 report published in JAMA Network Open said that use of an AI tool was associated with a higher agreement rate with dermatologists’ reference diagnoses for both PCPs and NPs.
In a 2021 blog post, Google scientists wrote that their AI model that powers a tool for checking skin conditions had earned European clearance, known as a CE mark, as a class I medical device.
SkinVision has an app that the company says “is available worldwide (with the exception of the USA and Canada).” The firm’s website includes a link where people in the United States and Canada can sign up for notifications about when SkinVision will be available in these nations.
‘Not ready for prime time’
The FDA panel did not cast formal votes at the July 29 meeting. Rather, the members engaged in broad discussions about risks and potential benefits of new tools for aiding in the detection of skin cancer.
Among the key issues discussed was a question of whether the FDA could impose requirements and restrictions, known as special controls, to provide “reasonable assurance of safety and effectiveness” for computer-aided devices that provide adjunctive diagnostic information to dermatologists about lesions suspicious for melanoma.
Among the potential special controls would be clinical performance testing in regards to rates of the sensitivity (true-positive rate) and specificity (true-negative rate).
The FDA could also look at requirements on software validation and verification and cybersecurity testing, as well as directions on labeling so as to mitigate risk.
Dermatologists serving on the panel called for caution in proceeding with steps that would make it easier for companies to market tools for aiding in melanoma diagnosis than it would be within the class III framework used for MelaFind and Nevisense.
Many expressed concerns about the need to design studies that would answer questions about how well new tools could accurately identify concerning lesions.
The phrase “not ready for prime time” was used at least three times during the discussion.
FDA panelist Maral Skelsey, MD, a skin cancer specialist from Chevy Chase, Maryland, said that over the years, she had used both Nevisense and MelaFind.
She said she had found MelaFind “unusable,” owing in large part to the high number of false positives it generated. The device also was limited as to where on patients’ bodies it could be used.
However, she spoke with enthusiasm about the prospects for better devices to aid in diagnosis of skin lesions. “It’s an area where we’re on the verge, and we really need these devices. There’s a need for patients to be able to examine themselves, for nondermatologists to be able to assess lesions,” Dr. Skelsey said.
But this field is “just not ready for prime time” yet, even with special controls, Dr. Skelsey said. To loosen approval standards too quickly could be a “detriment to what’s coming down the pipeline,” she said.
“It’s harmful to things that are likely to be around the corner,” she said.
FDA panelist Renata Block, PA-C, who works in a Chicago dermatology practice, pressed for maintaining a class III designation. “We are not ready for prime time yet, though the data that is coming down the pipeline on what we have is quite exciting,” Ms. Block said.
FDA panelist Karla V. Ballman, PhD, a statistician from Weill Cornell Medicine, New York, said there would need to be a clear standard for clinical performance before proceeding toward reclassification of devices for aid in detecting melanoma. “I just don’t think it’s ready for prime time at this point and should remain in class III,” she said.
But there was support from some panelists for the idea of a lower bar for clearance, combined with special controls to ensure patient safety.
In expressing her view, FDA panelist Katalin Roth, MD, JD, professor of medicine, George Washington University, Washington, said she was an outlier in her support for the agency’s view that these risks could be managed and that future tools could allow more patients to take a step on the pathway toward critical diagnoses.
“I deal with a lot of people with cancer as a palliative care physician,” Dr. Roth said. “I think what we’re missing here is the issue of time. Melanoma is a terrible disease, and missing the diagnosis is a terrible thing, but I think special controls would be sufficient to counter the concerns of my colleagues on the committee.”
The FDA’s Dr. Ashar ended the meeting with questions posed to one panelist, Veronica Rotemberg, MD, PhD, a dermatologist at Memorial Sloan Kettering Cancer Center in New York.
Dr. Rotemberg has for years been working in the field of research on developing AI and other computer-based tools for detecting and diagnosing melanoma, the deadliest form of skin cancer.
She has been publicly skeptical of the performance of commercial apps that scan moles and other lesions and that claim to identify which are cancerous. A May blog post on the Memorial Sloan Kettering website highlighted a recent British Journal of Dermatology article in which Dr. Rotemberg and coauthors reported on their evaluations of commercial apps. They judged them to be on average only 59% accurate, the blog post said.
However, during an earlier discussion at the meeting, she had spoken more positively about the prospects for using special controls in the near term to mitigate risk, although she said she would have a “very long list” of these requirements.
In the closing exchange with Dr. Ashar, Dr. Rotemberg outlined steps that could potentially ensure the safe use of tools to aid in melanoma screening. These included a need for postmarketing surveillance, which would require evaluation over time of algorithms used in tools meant to detect skin cancer.
“We need to have a mechanism for sampling,” Dr. Rotemberg said. “Most of our data is electronic now anyway, so comparing an algorithm and performance with biopsy results should not be that challenging.”
A version of this article first appeared on Medscape.com.
.
So far, the U.S. Food and Drug Administration has cleared two devices. Both are computer-aided skin lesion classification devices meant to help clinicians assess cases of suspected melanoma.
Both were given a class III designation. That classification is intended for products that are considered to have a high risk of harm because of flawed design or implementation. Many such devices are under development, and there has been a proposal to include these devices in class II, which is less restrictive.
The FDA turned to one of its expert panels for advice. At a meeting held on Aug. 29, experts on the panel offered differing views and expressed concerns about the accuracy of these devices.
This was the second day of meetings of the general and plastic surgery devices panel of the FDA’s Medical Devices Advisory Committee. On the previous day, the panel held a wide-ranging discussion about expanding use of skin lesion analyzer devices.
The FDA sought the expert panel’s advice concerning a field that appears to be heating up quickly after relatively quiet times.
Two devices have been approved by the FDA so far, but only one is still being promoted – SciBase AB’s Nevisense. The Swedish company announced in May 2020 that it had received FDA approval for Nevisense 3.0, the third generation of their Nevisense system for early melanoma detection, an AI-based point-of-care system for the noninvasive evaluation of irregular moles.
The other device, known as MelaFind, was acquired by Strata Skin Sciences, but the company said in 2017 that it discontinued research and development, sales, and support activity related to the device, according to a filing with the Securities and Exchange Commission.
But there’s been a swell in recent years in the number of publications related to the use of AI and machine learning, which could give rise to new tools for aiding in the diagnosis of skin conditions, including cancer. Google is among the companies that are involved in these efforts.
So, the FDA asked the expert panel to discuss a series of questions related to how the agency should weigh the risks of computer-aided devices for melanoma diagnosis. The agency also asked the panel to provide feedback about how well risks associated with such devices and tools might be managed and to offer suggestions.
The discussion at the July 29 meeting spun beyond narrow questions about reclassification of the current class III devices to topics involving emerging technology, such as efforts to apply AI to dermatology.
“Innovation continues. Medical device developers are anxious to plan how they might be able to develop the level of evidence that would meet your expectations” for future products, Binita Ashar, MD, a senior official in FDA’s Center for Devices and Radiological Health, told the panel.
Company CEO backs tougher regulation
Simon Grant, the chief executive of SciBase, which markets Nevisense, the first and only skin cancer–detecting device currently on the U.S. market, sought to make a case for sticking with the tougher class III regulations.
Speaking during the public comment session, Mr. Grant said switching to class II designations would weaken the standards used in clearing products that analyze skin lesions so as to put patients at risk.
Under the FDA’s rules, the agency designates as class III devices that present potential unreasonable risk of illness or injury. Only about 10% of devices fall into this category. Such devices include implantable pacemakers and breast implants, as well as SciBase’s Nevisense.
About 43% of medical devices fall into the class II category, which includes powered wheelchairs and some pregnancy test kits, the FDA website says.
Class I medical devices pose minimal potential for harm and tend to be simpler in design. These include enema kits and elastic bandages, the FDA says.
Mr. Grant told the meeting that in his career he has worked on two class III products and about 20 class II products. (He had previously worked at medical startups Synectics Medical and Neoventa, as well as established multinationals such as Medtronic.)
“I can tell you that – practically – the FDA has many fewer sticks and much less control when it comes to class II devices,” he said. He offered an example of a manufacturer of a class II device having more latitude in making small changes to products without notifying the FDA.
In his hypothetical example, such a change could have unintended consequences, and “with AI systems, small changes can result in large and nonlinear or even random effects,” Mr. Grant said. “But it’s too late if the product is on the market and the harm has already occurred,” he said.
The American Society for Dermatologic Surgery Association also protested the reclassifying of approved computer-aided melanoma detection class III devices.
In a statement posted on the FDA website as part of the materials for the meeting, the ASDSA raised a series of concerns about the prospects of expanded U.S. use of tools for assisting in diagnosing melanoma, including ones that would be marketed to consumers.
“To the extent that algorithms and devices for patient self-diagnosis of skin lesions are already widely available, they should be required to include detailed disclaimers that include that they are for entertainment and educational purposes and not a diagnostic device, that they are not approved by dermatologists or a recognized medical regulatory authority for self-diagnosis,” the ASDSA said.
Devices and algorithms in screening tools “are not highly regulated and remain unproven. They may result in wrong diagnoses, missed diagnoses, or over- or underdiagnosis,” the ASDSA added. “Both patients at low risk and those at high risk are better served by scheduling an in-person examination with a board-certified dermatologist, who can also help them determine the appropriate future skin screening schedule that is most appropriate for them.”
‘Stepping stone’
However, there is strong consumer demand for better information about skin conditions, and many patients face hurdles in going to dermatologists.
Google research has shown that consumers are seeking “a stepping stone” between the information they can easily find online and what they could get from a medical professional, said Lily Peng, MD, PhD, a director of product management for the health AI team at Google. Dr. Peng was a scheduled presenter at the July 29 meeting.
Consumers often are looking for more information on common conditions such as acne and poison ivy, and they sometimes face challenges in getting access to clinicians, she said.
“There are many unmet needs for consumers experiencing skin issues, many of which are lower-acuity conditions. There’s a big opportunity to increase accessibility and relevance of health journeys for consumers,” Dr. Peng said. “We have heard from consumers that they would like to have a self-help tool for nonserious conditions so they can decide when to seek medical attention.”
Dr. Peng’s presentation was not directly related to the question of class II or class III designation for existing products. Instead, her talk served as a glimpse into the work already underway in creating apps and tools for consumers.
Google researchers have published a number of studies in recent years about the use of AI to improve dermatology diagnosis.
A 2020 article reported on Google’s test of a form of AI known as deep learning system (DLS) to provide a differential diagnosis of skin conditions. On 963 validation cases, where a rotating panel of three board-certified dermatologists defined the reference standard, the DLS was noninferior to six other dermatologists and was superior to six primary care physicians (PCPs) and six nurse practitioners (NPs), according to a summary of the article.
A 2021 report published in JAMA Network Open said that use of an AI tool was associated with a higher agreement rate with dermatologists’ reference diagnoses for both PCPs and NPs.
In a 2021 blog post, Google scientists wrote that their AI model that powers a tool for checking skin conditions had earned European clearance, known as a CE mark, as a class I medical device.
SkinVision has an app that the company says “is available worldwide (with the exception of the USA and Canada).” The firm’s website includes a link where people in the United States and Canada can sign up for notifications about when SkinVision will be available in these nations.
‘Not ready for prime time’
The FDA panel did not cast formal votes at the July 29 meeting. Rather, the members engaged in broad discussions about risks and potential benefits of new tools for aiding in the detection of skin cancer.
Among the key issues discussed was a question of whether the FDA could impose requirements and restrictions, known as special controls, to provide “reasonable assurance of safety and effectiveness” for computer-aided devices that provide adjunctive diagnostic information to dermatologists about lesions suspicious for melanoma.
Among the potential special controls would be clinical performance testing in regards to rates of the sensitivity (true-positive rate) and specificity (true-negative rate).
The FDA could also look at requirements on software validation and verification and cybersecurity testing, as well as directions on labeling so as to mitigate risk.
Dermatologists serving on the panel called for caution in proceeding with steps that would make it easier for companies to market tools for aiding in melanoma diagnosis than it would be within the class III framework used for MelaFind and Nevisense.
Many expressed concerns about the need to design studies that would answer questions about how well new tools could accurately identify concerning lesions.
The phrase “not ready for prime time” was used at least three times during the discussion.
FDA panelist Maral Skelsey, MD, a skin cancer specialist from Chevy Chase, Maryland, said that over the years, she had used both Nevisense and MelaFind.
She said she had found MelaFind “unusable,” owing in large part to the high number of false positives it generated. The device also was limited as to where on patients’ bodies it could be used.
However, she spoke with enthusiasm about the prospects for better devices to aid in diagnosis of skin lesions. “It’s an area where we’re on the verge, and we really need these devices. There’s a need for patients to be able to examine themselves, for nondermatologists to be able to assess lesions,” Dr. Skelsey said.
But this field is “just not ready for prime time” yet, even with special controls, Dr. Skelsey said. To loosen approval standards too quickly could be a “detriment to what’s coming down the pipeline,” she said.
“It’s harmful to things that are likely to be around the corner,” she said.
FDA panelist Renata Block, PA-C, who works in a Chicago dermatology practice, pressed for maintaining a class III designation. “We are not ready for prime time yet, though the data that is coming down the pipeline on what we have is quite exciting,” Ms. Block said.
FDA panelist Karla V. Ballman, PhD, a statistician from Weill Cornell Medicine, New York, said there would need to be a clear standard for clinical performance before proceeding toward reclassification of devices for aid in detecting melanoma. “I just don’t think it’s ready for prime time at this point and should remain in class III,” she said.
But there was support from some panelists for the idea of a lower bar for clearance, combined with special controls to ensure patient safety.
In expressing her view, FDA panelist Katalin Roth, MD, JD, professor of medicine, George Washington University, Washington, said she was an outlier in her support for the agency’s view that these risks could be managed and that future tools could allow more patients to take a step on the pathway toward critical diagnoses.
“I deal with a lot of people with cancer as a palliative care physician,” Dr. Roth said. “I think what we’re missing here is the issue of time. Melanoma is a terrible disease, and missing the diagnosis is a terrible thing, but I think special controls would be sufficient to counter the concerns of my colleagues on the committee.”
The FDA’s Dr. Ashar ended the meeting with questions posed to one panelist, Veronica Rotemberg, MD, PhD, a dermatologist at Memorial Sloan Kettering Cancer Center in New York.
Dr. Rotemberg has for years been working in the field of research on developing AI and other computer-based tools for detecting and diagnosing melanoma, the deadliest form of skin cancer.
She has been publicly skeptical of the performance of commercial apps that scan moles and other lesions and that claim to identify which are cancerous. A May blog post on the Memorial Sloan Kettering website highlighted a recent British Journal of Dermatology article in which Dr. Rotemberg and coauthors reported on their evaluations of commercial apps. They judged them to be on average only 59% accurate, the blog post said.
However, during an earlier discussion at the meeting, she had spoken more positively about the prospects for using special controls in the near term to mitigate risk, although she said she would have a “very long list” of these requirements.
In the closing exchange with Dr. Ashar, Dr. Rotemberg outlined steps that could potentially ensure the safe use of tools to aid in melanoma screening. These included a need for postmarketing surveillance, which would require evaluation over time of algorithms used in tools meant to detect skin cancer.
“We need to have a mechanism for sampling,” Dr. Rotemberg said. “Most of our data is electronic now anyway, so comparing an algorithm and performance with biopsy results should not be that challenging.”
A version of this article first appeared on Medscape.com.
.
So far, the U.S. Food and Drug Administration has cleared two devices. Both are computer-aided skin lesion classification devices meant to help clinicians assess cases of suspected melanoma.
Both were given a class III designation. That classification is intended for products that are considered to have a high risk of harm because of flawed design or implementation. Many such devices are under development, and there has been a proposal to include these devices in class II, which is less restrictive.
The FDA turned to one of its expert panels for advice. At a meeting held on Aug. 29, experts on the panel offered differing views and expressed concerns about the accuracy of these devices.
This was the second day of meetings of the general and plastic surgery devices panel of the FDA’s Medical Devices Advisory Committee. On the previous day, the panel held a wide-ranging discussion about expanding use of skin lesion analyzer devices.
The FDA sought the expert panel’s advice concerning a field that appears to be heating up quickly after relatively quiet times.
Two devices have been approved by the FDA so far, but only one is still being promoted – SciBase AB’s Nevisense. The Swedish company announced in May 2020 that it had received FDA approval for Nevisense 3.0, the third generation of their Nevisense system for early melanoma detection, an AI-based point-of-care system for the noninvasive evaluation of irregular moles.
The other device, known as MelaFind, was acquired by Strata Skin Sciences, but the company said in 2017 that it discontinued research and development, sales, and support activity related to the device, according to a filing with the Securities and Exchange Commission.
But there’s been a swell in recent years in the number of publications related to the use of AI and machine learning, which could give rise to new tools for aiding in the diagnosis of skin conditions, including cancer. Google is among the companies that are involved in these efforts.
So, the FDA asked the expert panel to discuss a series of questions related to how the agency should weigh the risks of computer-aided devices for melanoma diagnosis. The agency also asked the panel to provide feedback about how well risks associated with such devices and tools might be managed and to offer suggestions.
The discussion at the July 29 meeting spun beyond narrow questions about reclassification of the current class III devices to topics involving emerging technology, such as efforts to apply AI to dermatology.
“Innovation continues. Medical device developers are anxious to plan how they might be able to develop the level of evidence that would meet your expectations” for future products, Binita Ashar, MD, a senior official in FDA’s Center for Devices and Radiological Health, told the panel.
Company CEO backs tougher regulation
Simon Grant, the chief executive of SciBase, which markets Nevisense, the first and only skin cancer–detecting device currently on the U.S. market, sought to make a case for sticking with the tougher class III regulations.
Speaking during the public comment session, Mr. Grant said switching to class II designations would weaken the standards used in clearing products that analyze skin lesions so as to put patients at risk.
Under the FDA’s rules, the agency designates as class III devices that present potential unreasonable risk of illness or injury. Only about 10% of devices fall into this category. Such devices include implantable pacemakers and breast implants, as well as SciBase’s Nevisense.
About 43% of medical devices fall into the class II category, which includes powered wheelchairs and some pregnancy test kits, the FDA website says.
Class I medical devices pose minimal potential for harm and tend to be simpler in design. These include enema kits and elastic bandages, the FDA says.
Mr. Grant told the meeting that in his career he has worked on two class III products and about 20 class II products. (He had previously worked at medical startups Synectics Medical and Neoventa, as well as established multinationals such as Medtronic.)
“I can tell you that – practically – the FDA has many fewer sticks and much less control when it comes to class II devices,” he said. He offered an example of a manufacturer of a class II device having more latitude in making small changes to products without notifying the FDA.
In his hypothetical example, such a change could have unintended consequences, and “with AI systems, small changes can result in large and nonlinear or even random effects,” Mr. Grant said. “But it’s too late if the product is on the market and the harm has already occurred,” he said.
The American Society for Dermatologic Surgery Association also protested the reclassifying of approved computer-aided melanoma detection class III devices.
In a statement posted on the FDA website as part of the materials for the meeting, the ASDSA raised a series of concerns about the prospects of expanded U.S. use of tools for assisting in diagnosing melanoma, including ones that would be marketed to consumers.
“To the extent that algorithms and devices for patient self-diagnosis of skin lesions are already widely available, they should be required to include detailed disclaimers that include that they are for entertainment and educational purposes and not a diagnostic device, that they are not approved by dermatologists or a recognized medical regulatory authority for self-diagnosis,” the ASDSA said.
Devices and algorithms in screening tools “are not highly regulated and remain unproven. They may result in wrong diagnoses, missed diagnoses, or over- or underdiagnosis,” the ASDSA added. “Both patients at low risk and those at high risk are better served by scheduling an in-person examination with a board-certified dermatologist, who can also help them determine the appropriate future skin screening schedule that is most appropriate for them.”
‘Stepping stone’
However, there is strong consumer demand for better information about skin conditions, and many patients face hurdles in going to dermatologists.
Google research has shown that consumers are seeking “a stepping stone” between the information they can easily find online and what they could get from a medical professional, said Lily Peng, MD, PhD, a director of product management for the health AI team at Google. Dr. Peng was a scheduled presenter at the July 29 meeting.
Consumers often are looking for more information on common conditions such as acne and poison ivy, and they sometimes face challenges in getting access to clinicians, she said.
“There are many unmet needs for consumers experiencing skin issues, many of which are lower-acuity conditions. There’s a big opportunity to increase accessibility and relevance of health journeys for consumers,” Dr. Peng said. “We have heard from consumers that they would like to have a self-help tool for nonserious conditions so they can decide when to seek medical attention.”
Dr. Peng’s presentation was not directly related to the question of class II or class III designation for existing products. Instead, her talk served as a glimpse into the work already underway in creating apps and tools for consumers.
Google researchers have published a number of studies in recent years about the use of AI to improve dermatology diagnosis.
A 2020 article reported on Google’s test of a form of AI known as deep learning system (DLS) to provide a differential diagnosis of skin conditions. On 963 validation cases, where a rotating panel of three board-certified dermatologists defined the reference standard, the DLS was noninferior to six other dermatologists and was superior to six primary care physicians (PCPs) and six nurse practitioners (NPs), according to a summary of the article.
A 2021 report published in JAMA Network Open said that use of an AI tool was associated with a higher agreement rate with dermatologists’ reference diagnoses for both PCPs and NPs.
In a 2021 blog post, Google scientists wrote that their AI model that powers a tool for checking skin conditions had earned European clearance, known as a CE mark, as a class I medical device.
SkinVision has an app that the company says “is available worldwide (with the exception of the USA and Canada).” The firm’s website includes a link where people in the United States and Canada can sign up for notifications about when SkinVision will be available in these nations.
‘Not ready for prime time’
The FDA panel did not cast formal votes at the July 29 meeting. Rather, the members engaged in broad discussions about risks and potential benefits of new tools for aiding in the detection of skin cancer.
Among the key issues discussed was a question of whether the FDA could impose requirements and restrictions, known as special controls, to provide “reasonable assurance of safety and effectiveness” for computer-aided devices that provide adjunctive diagnostic information to dermatologists about lesions suspicious for melanoma.
Among the potential special controls would be clinical performance testing in regards to rates of the sensitivity (true-positive rate) and specificity (true-negative rate).
The FDA could also look at requirements on software validation and verification and cybersecurity testing, as well as directions on labeling so as to mitigate risk.
Dermatologists serving on the panel called for caution in proceeding with steps that would make it easier for companies to market tools for aiding in melanoma diagnosis than it would be within the class III framework used for MelaFind and Nevisense.
Many expressed concerns about the need to design studies that would answer questions about how well new tools could accurately identify concerning lesions.
The phrase “not ready for prime time” was used at least three times during the discussion.
FDA panelist Maral Skelsey, MD, a skin cancer specialist from Chevy Chase, Maryland, said that over the years, she had used both Nevisense and MelaFind.
She said she had found MelaFind “unusable,” owing in large part to the high number of false positives it generated. The device also was limited as to where on patients’ bodies it could be used.
However, she spoke with enthusiasm about the prospects for better devices to aid in diagnosis of skin lesions. “It’s an area where we’re on the verge, and we really need these devices. There’s a need for patients to be able to examine themselves, for nondermatologists to be able to assess lesions,” Dr. Skelsey said.
But this field is “just not ready for prime time” yet, even with special controls, Dr. Skelsey said. To loosen approval standards too quickly could be a “detriment to what’s coming down the pipeline,” she said.
“It’s harmful to things that are likely to be around the corner,” she said.
FDA panelist Renata Block, PA-C, who works in a Chicago dermatology practice, pressed for maintaining a class III designation. “We are not ready for prime time yet, though the data that is coming down the pipeline on what we have is quite exciting,” Ms. Block said.
FDA panelist Karla V. Ballman, PhD, a statistician from Weill Cornell Medicine, New York, said there would need to be a clear standard for clinical performance before proceeding toward reclassification of devices for aid in detecting melanoma. “I just don’t think it’s ready for prime time at this point and should remain in class III,” she said.
But there was support from some panelists for the idea of a lower bar for clearance, combined with special controls to ensure patient safety.
In expressing her view, FDA panelist Katalin Roth, MD, JD, professor of medicine, George Washington University, Washington, said she was an outlier in her support for the agency’s view that these risks could be managed and that future tools could allow more patients to take a step on the pathway toward critical diagnoses.
“I deal with a lot of people with cancer as a palliative care physician,” Dr. Roth said. “I think what we’re missing here is the issue of time. Melanoma is a terrible disease, and missing the diagnosis is a terrible thing, but I think special controls would be sufficient to counter the concerns of my colleagues on the committee.”
The FDA’s Dr. Ashar ended the meeting with questions posed to one panelist, Veronica Rotemberg, MD, PhD, a dermatologist at Memorial Sloan Kettering Cancer Center in New York.
Dr. Rotemberg has for years been working in the field of research on developing AI and other computer-based tools for detecting and diagnosing melanoma, the deadliest form of skin cancer.
She has been publicly skeptical of the performance of commercial apps that scan moles and other lesions and that claim to identify which are cancerous. A May blog post on the Memorial Sloan Kettering website highlighted a recent British Journal of Dermatology article in which Dr. Rotemberg and coauthors reported on their evaluations of commercial apps. They judged them to be on average only 59% accurate, the blog post said.
However, during an earlier discussion at the meeting, she had spoken more positively about the prospects for using special controls in the near term to mitigate risk, although she said she would have a “very long list” of these requirements.
In the closing exchange with Dr. Ashar, Dr. Rotemberg outlined steps that could potentially ensure the safe use of tools to aid in melanoma screening. These included a need for postmarketing surveillance, which would require evaluation over time of algorithms used in tools meant to detect skin cancer.
“We need to have a mechanism for sampling,” Dr. Rotemberg said. “Most of our data is electronic now anyway, so comparing an algorithm and performance with biopsy results should not be that challenging.”
A version of this article first appeared on Medscape.com.