Multiple active stereo systems calibration method based on Neural SDF using DSSS for wide area 3D reconstruction

Kota Nishihara, Ryo Furukawa, Ryusuke Sagawa, Hiroshi Kawasaki; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 2838-2854

One of the practical methods for general 3D shape reconstruction is active stereo, which involves capturing images of scenes illuminated by structured light and reconstructing the scenes from the images. Active stereo has the advantage of being able to achieve high-density reconstructions even for texture-less scenes. However, it requires finding dense correspondences between the captured image and the projected pattern, making it difficult to capture the scene using multiple systems simultaneously where patterns are overlapped each other. A simple solution is to switch the patterns on and off synchronously; however, this naturally decreases exposure time in inverse proportion to the number of systems, resulting in a lower signal-to-noise ratio (SNR). In addition, retrieving relative poses between a projector and a camera of active stereo systems as well as relative poses between multiple systems is another challenge. To address these challenges, we propose a technique based neural signed distance field (Neural-SDF) using Direct Sequence Spread Spectrum (DSSS). DSSS is the latest technique widely used in the field of communications for multiplexing multiple signals and separating them. We propose a novel method to utilize DSSS to project multiple structured light patterns onto the object, where the patterns are overlapped and separated efficiently. To calibrate the relative poses between projectors and cameras in multiple sets of active stereo systems, a photometric loss-based calibration method using Neural-SDF is proposed. Such an approach has not yet been proposed for active stereo systems. In the experiments, it was proved that our technique worked successfully by both qualitative and quantitative evaluations using real sensors and objects.