End-to-End Deep Learning for Reconstructing Segmented 3D CT Image from Multi-Energy X-ray Projections

Siqi Wang, Tatsuya Yatagawa, Yutaka Ohtake, Toru Aoki, Jun Hotta; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 2574-2582

This paper presents an end-to-end deep-learning-based (DL-based) segmentation technique for multi-energy sparse-view CT, where local CT reconstruction and segmentation is achieved by a single network. While recent DL-based CT segmentation outperformed traditional methods in terms of accuracy and automation, these methods input a "reconstructed" CT, and thus, its performance highly depends on the CT image quality. The reliance prohibits the application of these techniques for sparse-view CT, whereas the sparse-view CT is another important technique to reduce radiation dose and image acquisition time. Our end-to-end deep learning technique integrates the reconstruction and segmentation within a single neural network, which allows us to improve the segmentation quality for sparse-view CT data. The proposed method extracts fragments of pixels from each multi-energy projection corresponding to a bar of CT image voxels. In this way, our network, comprising "filtering", "back-projection," and "segmentation" sub-networks, directly obtains the segmented CT image directly from projections. Our CT segmentation on a bar-by-bar basis is significantly memory-efficient due to the independence of memory-expensive 3D convolution. Consequently, our method delivers high-quality segmentation, where the problems of sparse-view artifacts and memory-expensiveness of prior methods are resolved.