SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilities

Kunato Nishina, Yusuke Matsui; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 8142-8147

Abstract

Text-to-image models have shown progress in recent years. Along with this progress generating vector graphics from text has also advanced. SVG is a popular format for vector graphics and SVG represents a scene with XML text. Therefore Large Language Models can directly process SVG code. Taking this into account we focused on editing SVG with LLMs. For quantitative evaluation of LLMs' ability to edit SVG we propose SVGEditBench. SVGEditBench is a benchmark for assessing the LLMs' ability to edit SVG code. We also show the GPT-4 and GPT-3.5 results when evaluated on the proposed benchmark. In the experiments GPT-4 showed superior performance to GPT-3.5 both quantitatively and qualitatively. The dataset is available at https://github.com/mti-lab/SVGEditBench.

Related Material

[pdf] [supp]
[bibtex]
@InProceedings{Nishina_2024_CVPR, author = {Nishina, Kunato and Matsui, Yusuke}, title = {SVGEditBench: A Benchmark Dataset for Quantitative Assessment of LLM's SVG Editing Capabilities}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2024}, pages = {8142-8147} }