Rescan: Inductive Instance Segmentation for Indoor RGBD Scans

Maciej Halber, Yifei Shi, Kai Xu, Thomas Funkhouser; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 2541-2550

Abstract


In depth-sensing applications ranging from home robotics to AR/VR, it will be common to acquire 3D scans of interior spaces repeatedly at sparse time intervals (e.g., as part of regular daily use). We propose an algorithm that analyzes these "rescans" to infer a temporal model of a scene with semantic instance information. Our algorithm operates inductively by using the temporal model resulting from past observations to infer an instance segmentation of a new scan, which is then used to update the temporal model. The model contains object instance associations across time and thus can be used to track individual objects, even though there are only sparse observations. During experiments with a new benchmark for the new task, our algorithm outperforms alternate approaches based on state-of-the-art networks for semantic instance segmentation.

Related Material


[pdf]
[bibtex]
@InProceedings{Halber_2019_ICCV,
author = {Halber, Maciej and Shi, Yifei and Xu, Kai and Funkhouser, Thomas},
title = {Rescan: Inductive Instance Segmentation for Indoor RGBD Scans},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}