The earlier a cancer is detected, the better the patient’s prognosis. Melanoma typically develops first on an accessible part of the body—the skin—and appears on the outer layer of the skin before spreading horizontally as well as deeper into the skin. In short, the earliest signs of melanoma can be detected in the clinic. So early detection of melanoma is feasible, and primary care physicians and dermatologists have “been trying to do the holy grail of catching and diagnosing melanoma early without the tradeoff of doing too many unnecessary biopsy procedures,” said Dr. Veronica Rotemberg, of Memorial Sloan Kettering Cancer Center, at the 2022 MRA Scientific Retreat.

But attaining that holy grail is challenging as detecting melanoma is currently based on subjective visual criteria and its accuracy varies greatly from practitioner to practitioner and clinic to clinic. “For lesions where it’s very important to catch them early and accurately, the gold standard for that is perhaps not as golden as we would like,” said Dr. Maria Wei of the University of California San Francisco. “Melanoma screening as currently conducted is not very precise.” It can also be difficult to access, particularly for racial minorities, people who live in rural areas, the elderly and those on Medicaid or lacking private insurance, she added. It can be challenging at times to get appointments with an expert dermatologist.

For all these reasons, researchers have developed artificial intelligence (AI) systems designed as tools to help doctors detect melanomas from photos of skin lesions. Initial testing of these systems was promising with several of them performing as well or even better than dermatologists evaluating the same photos. “Proof of principle doesn’t mean proof of practice and whether we can bring these techniques into the clinic with confidence,” noted Dr. Wei.

Room for Improvement and Refining Current AI Systems

She and Dr. Rotemberg pointed out several shortcomings of the currently available AI systems that suggest they aren’t ready for prime time, including their inaccuracy for nonpigmented lesions, for patients with skin of color, and for skin disorders other than those on which they had been trained. The latter is especially challenging considering there are close to 3,000 different skin conditions, but many AI melanoma detection systems have only been trained on a small subset of them, Dr. Wei noted, and they “try very hard to fit their diagnoses into diagnostic classes that they know,” she said.

Their accuracy also declines when they confront minor differences among images, such as changes in lighting, contrast, or even the positioning. The AI may accurately detect a melanoma under the ‘perfect conditions’ or when an image is in one position, but may miss the diagnosis when there is shadowing, hair, or if the image is turned upside down. This can lead not only to missing a melanoma when it occurs, but to unnecessary biopsies due to misdiagnosis. For example, nearly half the time AI confused scars with basal cell carcinoma, which may have necessitated a biopsy for confirmation.

The accuracy of AI systems can also vary from one clinic to another due to differences in how common melanoma is in each, or whether the clinic has a diverse patient population appropriately represented in the images used to train the AI system. Dr. Rotemberg noted. “I don’t want to say that this is scary or get people worried, but I just want us to think about the information we need when we’re trying to adjudicate whether AI should be used more widely,” she said.

She added that these AI systems may be substantially less accurate when confronted with rare melanomas, such as pediatric melanomas, which tend to look different than typical melanomas. And both she and Dr. Wei noted that the accuracy of AI systems may not be the same in minority populations underrepresented in the public data sets used to develop and train currently available AI systems. “It’s possible you won’t be able to take an off-the-shelf computer model and use it on your entire population. You may have to supplement its training with the subpopulation(s) you’ll be using the model on,” Dr. Wei said.

AI systems lose their accuracy when images are blurred, unlike dermatologists, but they also seem to detect features that humans are not observing. “The fact that AI is evaluating lesions differently than doctors suggests that computer vision can augment provider diagnostics,” said Dr. Wei. She offers that rather than replacing dermatologists, AI melanoma detection systems will be used in conjunction with physician evaluations of suspicious lesions. “AI-based technologies can help us distill information to more useable degrees and to make more valid clinical decisions on that basis,” said Dr. Wei.

The Future of AI in Melanoma

Drs. Wei and Rotemberg suggested several ways to improve AI melanoma detection systems, including expanded public image datasets that include a greater variety of ethnicities, races, and ages. Dr. Wei is partnering with the San Francisco General Hospital to create a dataset that’s much more diverse than those currently available. Dr. Rotemberg also suggested including more images in such datasets from rare melanoma subtypes and those taken from unusual anatomic sites.

“The horizon is really bright for improving AI tools for doctors to use, with a lot coming down the pipeline that can bring screening of melanoma into precision medicine and also address gaps in care with racial minorities and access issues,” said Dr. Wei. “It’s just not there yet.”

This post was originally published November 15, 2022, by the Melanoma Research Alliance. It is republished with permission.