Panoptic segmentation of nUclei and tissue in advanced MelanomA¶

We are pleased to announce the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) challenge, a histopathology grand challenge aimed at improving nuclei and tissue segmentation in melanoma histopathology. This challenge addresses the need for improved prognostic biomarkers to better predict treatment responses in melanoma patients.

Background: Advanced Melanoma

Melanoma is an aggressive form of skin cancer with increasing incidence. While primary melanoma is often treated with surgical excision, advanced melanoma requires immune checkpoint inhibition therapy. Unfortunately, half of the patients do not respond to this therapy, which is costly and potentially toxic [1], [2].

The Need For Prognostic Biomarkers

Previous research has shown that the presence of tumor infiltrating lymphocytes (TILs) before treatment is associated with better responses to therapy and improved survival rates [3], [4]. However, manual methods for scoring TILs are subjective and inconsistent [3].

In addition, not only the presence of TILs might be prognostic. Localization (intratumoral, peritumoral or stromal) and other immune cell subsets such as neutrophils or plasma cells might also hold predictive value [5], [6].

Current clinically validated deep learning methods rely on older techniques resulting in low performance [4], [7]. In addition, other models such as HoverNet trained on the PanNuke dataset (which contains skin examples) also have suboptimal performance [8], [9]. This is due to the melanoma specific ability of mimicking other cell types, leading to misclassifications such as identifying tumors as stroma or lymphocytes, or epidermal skin cells as tumors.

Consequently, there is a need for a well-performing deep learning model capable of segmenting nuclei of tumor cells and different immune cell subsets in melanoma histopathology. In addition, there is need for a model capable of segmenting tissue types which can be used for cell localization and circumventing common misclassification issues. To address these needs, we created the Panoptic Segmentation of nUclei and tissue in advanced MelanomA (PUMA) dataset.

The PUMA dataset includes:

• 155 primary and 155 metastatic melanoma regions of interest (ROI), scanned at 40x magnification with a resolution of 1024 x 1024 pixels.
• Context ROI of 5120 x 5120 pixels, centered around each ROI.
• Annotations for tissue and nuclei, provided by a medical professional and verified by a board-certified dermatopathologist.

The Challenge Tasks

​The PUMA Challenge consists of two tracks, each with two tasks.

• Track 1 – Panoptic segmentation with three instance classes:

Task 1: Semantic tissue segmentation of tumor, stroma, epithelium, blood vessel, and necrotic regions.

Task 2: Nuclei detection for three classes; tumor, TILs (lymphocytes and plasma cells), and other cells (histiocytes, melanophages, neutrophils, stromal cells, epithelium, endothelium, and apoptotic cells).

• Track 2 – Panoptic segmentation with ten instance classes:

Task 1: Semantic tissue segmentation of tumor, stroma, epithelium, blood vessel, and necrotic regions.

Task 2: Nuclei detection for all classes: tumor, lymphocytes, plasma cells, histiocytes, melanophages, neutrophils, stromal cells, epithelium, endothelium, and apoptotic cells.

Participants can join the challenge in one or both tracks, using one or multiple task specific models.

NB. Task 1 is the same in track 1 and track 2.

References:

[1] Franken, M. G. et al. Trends in survival and costs in metastatic melanoma in the era of novel targeted and immunotherapeutic drugs. ESMO Open 6, 100320 (2021). Link

[2] Postow, M. A., Sidlow, R. & Hellmann, M. D. Immune-Related Adverse Events Associated with Immune Checkpoint Blockade. N Engl J Med 378, 158–168 (2018). Link

[3] van Duin, I. A. J. et al. Baseline tumor-infiltrating lymphocyte patterns and response to immune checkpoint inhibition in metastatic cutaneous melanoma. European Journal of Cancer 114190 (2024) doi:10.1016/j.ejca.2024.114190. Link

[4] Chatziioannou, E. et al. Deep learning-based scoring of tumour-infiltrating lymphocytes is prognostic in primary melanoma and predictive to PD-1 checkpoint inhibition in melanoma metastases. eBioMedicine 93, (2023). Link

[5] Park, S. et al. Artificial Intelligence–Powered Spatial Analysis of Tumor-Infiltrating Lymphocytes as Complementary Biomarker for Immune Checkpoint Inhibition in Non–Small-Cell Lung Cancer. JCO 40, 1916–1928 (2022). Link

[6] Helmink, B. A. et al. B cells and tertiary lymphoid structures promote immunotherapy response. Nature 577, 549–555 (2020). Link

[7] Schuiveling, M. et al. A Novel Dataset for Nuclei Segmentation in Melanoma Histopathology. MIDL short paper (2024). Link

[8] Graham, S. et al. Hover-Net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Analysis 58, 101563 (2019). Link

[9] Gamper, J. et al. PanNuke Dataset Extension, Insights and Baselines. Preprint at http://arxiv.org/abs/2003.10778 (2020).