Pigment Mapping for Tomb Murals Using Neural Representation and Physics-Based Model

Mayuka Tsuji, Yuki Fujimura, Takuya Funatomi, Yasuhiro Mukaigawa, Tetsuro Morimoto, Takeshi Oishi, Jun Takamatsu, Katsushi Ikeuchi; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 1671-1679

This paper presents a pigment mapping in tomb murals for digitization. In order to separate pigments from their substrates, we utilize the Kubelka-Munk (KM) model. However, these murals are drawn on rocks, and the pigments have deteriorated and thinned over time. As such, the challenge is to cancel the impact of the rocks' heterogeneous patterns; previous studies using the KM model either ignored the substrate or assumed it to be constant. We introduce unsupervised learning based on neural representations and physics to facilitate pigment mapping, even on a heterogeneous substrate. The model takes an image of the spectral reflectance data at a specific position of a tomb mural image and the corresponding position as inputs and outputs the pigment thickness, pigment class, and substrate class. For physically-consistent estimation, the input reflectance is reconstructed using the Kubelka-Munk model and the output. This allows unsupervised training via the calculation of the reconstruction loss. While the Kubelka-Munk model operates on a pixel-by-pixel basis, the utilization of neural representation by the input position facilitates highly accurate estimation, all the while maintaining spatial continuity.