Reference Grid-assisted Network for 3D Point Signature Learning from Point Clouds

Jing Zhu, Yi Fang; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2020, pp. 211-220

Abstract


Learning a robust 3D point signature from point clouds is an interesting but challenging task in the computer vision field due to the irregular and unordered structure characteristics of the point cloud data. In this paper, we propose to learn a 3D point signature by exploring the implicit relation between keypoints and their neighbors (grouped as patches) among the given scene point clouds. Specially, we design a uniform reference grid to represent the raw relation between each keypoint and its neighbors from the raw point clouds. In order to learn a 3D point signature gradually from a smaller perceptive region to a larger area, we create a novel framework with a MLP-based unit feature network and a 3D CNN-based grid feature network. Specifically, the unit feature network aims to dig the connections from points fallen into the same unit of the reference grid, while the grid feature network is used to discover the grid-wise relations across the whole reference grid with concatenation of the learned unit-wise features. Moreover, we introduce an MLP-based attention network upon the unit feature network to enhance the discriminative ability of our learned 3D point signature. All the components in our proposed model are implemented as siamese ones to better tackle the classic keypoint matching and geometric registration problems. Our proposed 3D point signature learning approach achieves superior performance over other state-of-the-art methods on keypoint matching and geometric registration on the real-world scenes datasets, e.g. SUN3D, 7-scenes and the synthetic scan augmented scenes in ICL-NUIM dataset. More importantly, our learned 3D point signature successfully handles the point cloud fragment alignment challenges by producing correct transformations with RANSAC algorithm.

Related Material


[pdf] [video]
[bibtex]
@InProceedings{Zhu_2020_WACV,
author = {Zhu, Jing and Fang, Yi},
title = {Reference Grid-assisted Network for 3D Point Signature Learning from Point Clouds},
booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
month = {March},
year = {2020}
}