09 | Integration of hematoxylin and eosin and immunohistochemistry images to improve AI-based recurrence prediction in negative sentinel lymphnode melanoma patients
Maria Colomba Comes1, Andrea Lupo1, Fabio Mele2, Concetta Saponaro2, Arianna Bozzi1, Ivana De Risi3, Samantha Bove1, Annarita Fanizzi1, Benedetta Apollonio3, Margherita Sonnessa2, Sabino Strippoli3, Francesco Alfredo Zito2, Michele Guida3, Raffaella Massafra1 | 1Biostatistics and Bioinformatics Unit, IRCCS Istituto Tumori "Giovanni Paolo II"; 2Pathology Unit, IRCCS Istituto Tumori "Giovanni Paolo II"; 3Rare Tumors and Melanoma Unit, IRCCS Istituto Tumori "Giovanni Paolo II", Bari, Italy.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
Background: Stage IB–IIC melanoma with negative sentinel lymph node (SLN) status shows heterogeneous recurrence risk. Hematoxylin and eosin (H&E) slides capture tissue morphology, while immunohistochemistry (IHC) assays highlight immune, proliferative, and lymphvascular features. To date, no artificial intelligence (AI)-based model has combined these modalities for recurrence prediction in melanoma.
Methods: We analyzed tumor tissues from 68 SLN-negative patients treated at IRCCS Istituto Tumori of Bari (Table 1, 29.4% recurrence rate within 2 years). For each sample, regions of interest (ROIs) were annotated by pathologists on H&E slides, distinguishing TUMOR areas (tumor tissue only) and TUMOR+INF areas (tumor and surrounding infiltrating cells). Corresponding IHC slides, including β-catenin, CD4-CD8, CD20, CD64, CD163, Ki-67, FOXP3, and podoplanin, were co-registered to the H&E reference (Figure 1). From these ROIs, ~3,000 TUMOR tiles and TUMOR+INF tiles (296×296 px at 40× magnification) were automatically extracted. An AI-based approach was implemented using a multi-channel convolutional neural network that jointly processes co-registered stains, capturing pixel-level spatial correspondences. Two distinct models were trained: one for TUMOR ROIs and another for TUMOR+INF ROIs. Performances were evaluated according to a 5-fold cross-validation scheme.
Results: Compared with our previous H&E-only approach [1], where AUCs were 0.791 for TUMOR and 0.623 for TUMOR+INF, the integration of IHC features markedly improved predictive performance in both ROI categories. With the multimodal model, the TUMOR-only analysis achieved an AUC of 0.850 (95%CI [0.738–0.945]), while the TUMOR+INF model reached 0.926 (95%CI [0.852–0.979]). The largest performance gain was observed in TUMOR+INF regions, underscoring the added value of immune and microenvironmental features that are not adequately captured by H&E morphology alone.
Conclusions: Multimodal integration of H&E and IHC enhances the predictive strength of AI-based models by uncovering insights that are less discernible on morphology alone.
References:
[1] https://doi.org/10.1186/s12967-024-05629-2
Table 1.
Figure 1.
Downloads
Citations
How to Cite
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
