Semantic Segmentation Method for Automated Indoor 3D Reconstruction Based on Architectural-Knowledge-Aware Features

Yahan Chen, Wenzheng Liu, Xiaowei Luo; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 2715-2724

3D point cloud semantic segmentation is an important step for 3D indoors reconstruction. In recent years many outstanding deep learning models have been proposed for semantic segmentation which can achieve remarkable performance. However it is found the index indicating semantic prediction accuracy in terms of structural components (e.g. columns beams etc.) in buildings is far from satisfying lacking sufficient information for subsequent 3D reconstruction. For better segmenting and identifying structural components this work proposes Architectural-Knowledge-Aware features (AKAFs) i.e. F1 and F2 which are strategically incorporated into a developed two-stage training framework wherein outstanding semantic segmentation models are adopted as backbones. By incorporating F1 which formalizes the position distribution pattern of building structural components semantic information can be explored preliminary in the explicit stage (i.e. the first stage). The second stage is the implicit stage where F2 is derived based on the Semantic and Relative Position Fusing Module (SRPFM) which implicitly introduces more relative position information of building components for semantic segmentation. Extensive experiments have been conducted on the S3DIS dataset adopting three outstanding backbones. Results demonstrate that the proposed AKAFs can increase the accuracy of segmentation for structural components by more than 5%. As a result the overall semantic segmentation performance increases by 2%. Even on the current SOTA model (PT-v3) the proposed AKAFs show promising semantic segmentation promotion. Portable and well-compatible with most point-based semantic segmentation models AKAFs can effectively perceive more architectural characteristics hidden in indoor point clouds especially prominent for points of structural components.