DeFi: Detection and Filling of Holes in Point Clouds Towards Restoration of Digitized Cultural Heritage Models

Ramesh Ashok Tabib, Dikshit Hegde, Tejas Anvekar, Uma Mudenagudi; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 1603-1612

In this paper, we propose DeFi: a novel perspective for hole detection and filling of a given deteriorated 3D point cloud towards digital preservation of cultural heritage sites. Preservation of heritage demands digitization as cultural heritage sites deteriorate due to natural calamities and human activities. Digital preservation promotes acquisition of 3D data using 3D sensor or Multi-view reconstruction. Unfortunately, 3D data acquisition finds challenges due to the limitations in sensor technology and inappropriate capture conditions, leading to formation of missing regions or holes in the acquired point cloud. To address this, we propose a pipeline consisting of detection of hole boundaries, and understanding the geometry of the hole boundaries to fill the region of the point cloud. Recent research on hole detection and filling fails to generalize on complex structures such as heritage sites, as they find challenges in differentiating between the hole boundary and non-hole boundary points. To address this, we propose to detect boundary points of point cloud and learn to classify them into "hole boundary" and "non-hole boundary" points. We generate a synthetic dataset based on ModelNet40 to learn the detection of hole boundaries. We demonstrate the results of the proposed pipeline on (i) ModelNet40 dataset, (ii) Heritage 3D models generated via photogrammetry, and compare the results with state-of-the-art methods.