3D Shape Completion using Multi-Resolution Spectral Encoding

Pallabjyoti Deka, Saumik Bhattacharya, Debashis Sen, Prabir Kumar Biswas; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 54-63

Reconstruction of intricate local patterns and large missing regions during 3D shape completion has the contradictory requirements of computation over a wider context and operations for finer detail restoration. To this end we propose a multi-resolution spectral encoding based 3D shape completion approach to work on truncated Signed Distance Field (SDF) based shape representations. Our novelty lies in judiciously integrating multi-resolution 3D convolutional blocks that encode the input shape and a spectral module (SM) that captures the shape-wide context thus addressing the contradictory requirements. SM acts on the features extracted from both partial input scans and shape priors using the multi-resolution convolutional blocks. Our SM contains a 3D convolutional block placed between fast Fourier transform (FFT) and inverse FFT operations which results in the expansion of the receptive field for the appropriate context computation. Our approach has an attention-based encoder-decoder architecture where the encoding of a partial scan is acted upon by shape prior encodings to produce attention maps. These attention maps are leveraged differently in pretraining and in the later training and inference stages of our approach to produce the reconstructed 3D shape. A surface gradient-based loss function is used in addition to the L1 loss both in the pretraining and training stages for emphasizing the differences in minute details. These along with an attention refinement operation often leads to complete reconstruction while restoring finer details. Experiments using standard synthetic and real datasets demonstrate the superiority of our approach over the state-of-the-art.