Diagram Image Retrieval and Analysis: Challenges and Opportunities

Liping Yang, Ming Gong, Vijayan K. Asari; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 180-181

Deep learning has achieved significant advances for tasks such as image classification, segmentation, and retrieval; this advance has not yet been realized on scientific and technical drawing images. Research for technical diagram image analysis and retrieval retain much less well developed compared to natural images; one major reason is that the dominant features in scientific diagram images are shape and topology, no color and intensity features, which are essential in retrieval and analysis of natural images. One important purpose of this review, along with some challenges and opportunities, is to draw the attention of researchers and practitioners in the Computer Vision community to the strong needs of advancing research for diagram image retrieval and analysis, beyond the current focus on natural images, in order to move machine vision closer to artificial general intelligence. This paper investigates recent research on diagram image retrieval and analysis, with an emphasis on methods using content-based image retrieval (CBIR), textures, shapes, topology and geometry. Based on our systematic review of key research on diagram image retrieval and analysis, we then demonstrate and discuss some of the main technical challenges to be overcome for diagram image retrieval and analysis, and point out future research opportunities from technical and application perspectives.