Fine-tuning Large Language Models for Automatic Font Skeleton Generation: Exploration and Analysis

Yuxuan Liu, Yasuhisa Fujii, Xinru Zhu, Kayoko Nohara; Proceedings of the Asian Conference on Computer Vision (ACCV), 2024, pp. 4156-4171

Despite the pivotal role that font skeletons could play in typeface research and font design, the availability of font skeleton data is sparse and limited. This research explores the possibility of using Large Language Models (LLMs) to generate font skeleton data based on font outline data. Our method represents font skeletons and font outlines as sequences of text tokens derived from SVG commands, and formulates the font skeleton task as a language modeling task predicting the token sequence for a font skeleton given the token sequence of a font outline. As a first attempt, we fine-tuned GPT-3.5 on a dataset of 8,213 Japanese font outlines and corresponding skeletons. Both quantitative and qualitative evaluations show the effectiveness of the approach in terms of rasterized pixel distance, Chamfer distance, and visual analysis. Our proposed method achieved average results of 14.678 for rasterized pixel distance and 1.713 for Chamfer distance, both better than the baseline method (PolyVectorization). In visual analysis, we found better generation results for complex shapes which logograms such as Chinese characters tend to have, than for the simple shapes of syllabograms such as Japanese kana, phonograms such as Latin alphabets, and symbols. Although our fine-tuned model has limitations in generating the skeletons of other font styles, this research establishes a foundation for the automatic generation of font skeletons using LLMs, setting the stage for future work on automatic skeleton generation and the wider application of font skeletons in typography.