Multi-View Coupled Self-Attention Network for Pulmonary Nodules Classification

Qikui Zhu, Yanqing Wang, Xiangpeng Chu, Xiongwen Yang, Wenzhao Zhong; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 995-1009


Evaluation of the malignant degree of pulmonary nodules plays an important role in early detecting lung cancer. Deep learning-based methods have obtained promising results in this domain with their effectiveness in learning feature representation. Both local and global features are crucial for medical image classification tasks, particularly for 3D medical image data, however, the receptive field of convolution kernel limits the global feature learning. Although self-attention mechanism can success fully model long-range dependencies by directly flattening the input image to a sequence, which has high computational complexity. Additionally, which unable to model the image local context information across spatial and depth dimensions. To address the above challenges, in this paper, we carefully design a Multi-View Coupled Self-Attention Module (MVCS). Specifically, a novel self-attention module is proposed to model spatial and dimensional correlations sequentially for learning global spatial contexts and further improving the identification accuracy. Compared with vanilla self-attention, which have three-fold advances: 1) uses fewer memory consumption and computational complexity than the existing self-attention methods; 2) except for exploiting the correlations along the spatial and channel dimension, the dimension correlations are also exploited; 3) the proposed self-attention module can be easily integrated with other frameworks. By adding the proposed module into 3D ResNet50, we build a classification network for lung nodules' malignancy evaluation. The nodule classification network was validated on a public dataset from LIDC-IDRI. Extensive experimental results demonstrate that our proposed model has performance comparable to state-of-the-art approaches.

Related Material

[pdf] [code]
@InProceedings{Zhu_2022_ACCV, author = {Zhu, Qikui and Wang, Yanqing and Chu, Xiangpeng and Yang, Xiongwen and Zhong, Wenzhao}, title = {Multi-View Coupled Self-Attention Network for Pulmonary Nodules Classification}, booktitle = {Proceedings of the Asian Conference on Computer Vision (ACCV)}, month = {December}, year = {2022}, pages = {995-1009} }