Recognizing Handwritten Mathematical Expressions via Paired Dual Loss Attention Network and Printed Mathematical Expressions

Anh Duc Le; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 566-567

Recognition of Handwritten Mathematical Expressions (HMEs) is a challenging problem because of the complicated structure and uncommon math symbols contained in HMEs. Moreover, the lack of training data is a serious issue, especially for deep learning-based systems. In this paper, we proposed a dual loss attention model that utilizes the existing latex corpus to improve accuracy. The proposed dual loss attention has two losses, including decoder loss and context matching loss to learn semantic invariant features for the encoder and latex grammar for the decoder from handwritten and printed MEs. The results of experiments on the CROHME 2014 and 2016 databases demonstrate the superiority and effectiveness of our proposed model. These results are competitive compared to others reported in recent literature.