Continuous Spatio-Temporal Memory Networks for 4D Cardiac Cine MRI Segmentation

Meng Ye, Bingyu Xin, Leon Axel, Dimitris Metaxas; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9514-9524

Current cardiac cine magnetic resonance image (cMR) studies focus on the end diastole (ED) and end systole (ES) phases while ignoring the abundant temporal information in the whole image sequence. This is because whole sequence segmentation is currently a tedious process and inaccurate. Conventional whole sequence segmentation approaches first estimate the motion field between frames which is then used to propagate the mask along the temporal axis. However the mask propagation results could be prone to error especially for the basal and apex slices where through-plane motion leads to significant morphology and structural change during the cardiac cycle. Inspired by recent advances in video object segmentation (VOS) based on spatio-temporal memory (STM) networks we propose a continuous STM (CSTM) network for semi-supervised whole heart and whole sequence cMR segmentation. Our CSTM network takes full advantage of the spatial scale temporal and through-plane continuity prior of the underlying heart anatomy structures to achieve accurate and fast 4D segmentation. Results of extensive experiments across multiple cMR datasets show that our method can improve the 4D cMR segmentation performance especially for the hard-to-segment regions. Project page is at https://github.com/DeepTag/CSTM.