Font-ProtoNet: Prototypical Network-Based Font Identification of Document Images in Low Data Regime

Nikita Goel, Monika Sharma, Lovekesh Vig; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020, pp. 556-557

While optical character recognition has attracted considerable interest from researchers in recent times, automating font identification in printed / scanned documents is still not a well explored problem. With the increasing variety of fonts in the open community, identifying the different fonts used in a given document image can often provide important visual cues for document understanding. Font identification is a challenging task owing to smaller inter-class variations and limited availability of labeled image data for a large variety of font images. In the absence of the original true type format (ttf) files, even synthetic data generation is not possible. To this end, we propose to utilize recent few-shot learning techniques like prototypical networks for font identification in scanned / printed document images using character images from different fonts as input for scarce data scenarios and call the proposed method Font-ProtoNet. This approach uses an initial set of classes to learn an embedding and centroid representations (as class prototypes), that are used to classify novel samples based on euclidean distance. We demonstrate that Font-ProtoNet gives encouraging results by training prototypical networks in few-shot learning settings on a synthetic dataset of 200 font classes and using the trained network to identify fonts on a synthetic dataset of 100 novel font classes. We have also tested our approach on the real-world Adobe Visual Font Recognition (AdobeVFR) dataset and obtained 59.86% and 71.01% word-level accuracy of font identification using 1-shot and 5-shot i.e.,1 and 5 character images per font class, respectively.