-
[pdf]
[supp]
[arXiv]
[bibtex]@InProceedings{Zhang_2023_CVPR, author = {Zhang, Juze and Luo, Haimin and Yang, Hongdi and Xu, Xinru and Wu, Qianyang and Shi, Ye and Yu, Jingyi and Xu, Lan and Wang, Jingya}, title = {NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2023}, pages = {8834-8845} }
NeuralDome: A Neural Modeling Pipeline on Multi-View Human-Object Interactions
Abstract
Humans constantly interact with objects in daily life tasks. Capturing such processes and subsequently conducting visual inferences from a fixed viewpoint suffers from occlusions, shape and texture ambiguities, motions, etc. To mitigate the problem, it is essential to build a training dataset that captures free-viewpoint interactions. We construct a dense multi-view dome to acquire a complex human object interaction dataset, named HODome, that consists of 71M frames on 10 subjects interacting with 23 objects. To process the HODome dataset, we develop NeuralDome, a layer-wise neural processing pipeline tailored for multi-view video inputs to conduct accurate tracking, geometry reconstruction and free-view rendering, for both human subjects and objects. Extensive experiments on the HODome dataset demonstrate the effectiveness of NeuralDome on a variety of inference, modeling, and rendering tasks. Both the dataset and the NeuralDome tools will be disseminated to the community for further development, which can be found at https://juzezhang.github.io/NeuralDome
Related Material