HiFormer: Hierarchical Multi-Scale Representations Using Transformers for Medical Image Segmentation

Moein Heidari, Amirhossein Kazerouni, Milad Soltany, Reza Azad, Ehsan Khodapanah Aghdam, Julien Cohen-Adad, Dorit Merhof; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 6202-6212

Abstract


Convolutional neural networks (CNNs) have been the consensus for medical image segmentation tasks. However, they inevitably suffer from the limitation in modeling long-range dependencies and spatial correlations due to the nature of convolution operation. Although Transformers were first developed to address this issue, they fail to capture low-level features. In contrast, it is demonstrated that both local and global features are crucial for dense prediction, such as segmenting in challenging contexts. In this paper, we propose HiFormer, a novel method that efficiently bridges a Convolutional neural network and a Transformer for medical image segmentation. Specifically, we design two multi-scale feature representations using the seminal Swin-Transformer module and a CNN-based encoder. To secure a fine fusion of global and local features obtained from the two aforementioned representations, we propose a Double-Level Fusion (DLF) module in the skip connection of the encoder-decoder outline. Extensive experiments on various medical image segmentation datasets demonstrate the effectiveness of HiFormer over other CNN-based, Transformer-based, and hybrid methods in terms of computational complexity, quantitative and qualitative results

Related Material


[pdf] [supp] [arXiv]
[bibtex]
@InProceedings{Heidari_2023_WACV, author = {Heidari, Moein and Kazerouni, Amirhossein and Soltany, Milad and Azad, Reza and Aghdam, Ehsan Khodapanah and Cohen-Adad, Julien and Merhof, Dorit}, title = {HiFormer: Hierarchical Multi-Scale Representations Using Transformers for Medical Image Segmentation}, booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, month = {January}, year = {2023}, pages = {6202-6212} }