Generalizable Structure-Aware INF: Biplanar-View CT Reconstruction via Disentangled Implicit Neural Field

Bei Huang, Yuru Pei; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 699-715

Structure-aware CT reconstruction from a single or biplanar X-rays produces patient-specific 3D insights into underlying structures, pushing the radiation hazards during diagnosis and treatment to a minimum. Existing implicit neural fields (INF) methods have shown impressive performance in CT reconstruction, though they rely on multi-view X-rays and conduct subject-specific reconstruction. Additional online adaptation is required to handle novel subjects. In this paper, we present the generalizable structure-aware implicit neural fields (GSA-INF), a unified model that learns a generalizable structure-aware volume prior to INF decoding from sparse-view X-rays. Previous CoderNeRF views the latent code as a holistic shape prior to 3D reconstruction. In contrast, we present a new triplane generative model to learn a generalizable volume prior distribution, where the sampled triplane latent code produces voxel-level representation for INF decoding and CT reconstruction. Moreover, we introduce anatomical structure mask supervision by building a parallel INF-based decoding framework that enhances structure disentanglements when popping up a variety of structures from 2D X-rays. Our approach entails simultaneous INF-based CT reconstruction and volume-prior learning. In the online inference process, we can conditionally reconstruct CT from single- or biplanar-view X-rays and unconditionally generate CTs via sampling in the latent space. GSA-INF demonstrates robust and superior results over the compared methods.