Multi-path Segmentation Network Based on CNN and Transformer for Skin Lesion Image

Tianyu Nie, Yishi Zhao, Shihong Yao; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 3137-3153

Skin lesion segmentation is a challenging task in computer-aided di-agnosis, which is crucial for the early diagnosis of skin cancer. Convolutional Neural Networks (CNNs) have been successful in medical image segmentation tasks; however, their effective receptive fields in deep convolutional layers are limited to a local range and follow Gaussian distribution, thereby failing to ob-tain global information. Advanced Transformer shows great potential in model-ing long-range dependencies and obtaining global representations. Therefore, we propose a multi-path segmentation model (MSNet) based on a combination of CNN and Transformer, which is dedicated to facilitating the task of skin le-sion segmentation. Regarding different task requirements, we design MSNet-1 for the real-time tasks, and MSNet-2 for the tasks that require high accuracy. Moreover, we develop an efficient residual module (ERM) in MSNet, which can effectively integrate multi-level features and provide accurate feature repre-sentations. Pixel attention and coordinate attention are also introduced to en-hance the perceptual ability of the network and improve the predicting accuracy of the segmentation results. Finally, we conduct extensive experiments on three public skin lesion datasets and one thyroid nodule dataset. The experimental re-sults demonstrate that MSNet not only possesses the SOTA segmentation per-formance and excellent generalization ability, but also has lightweight and real-time characteristics, and it has broad application prospects in various scenarios.