GELDA: A Generative Language Annotation Framework to Reveal Visual Biases in Image Generators

Krish Kabra, Kathleen M. Lewis, Guha Balakrishnan; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 8304-8309

Abstract


In this work we propose GELDA an automatic framework that leverages large language models (LLMs) and vision-language models (VLMs) to reveal visual biases in image generators. GELDA takes a user-defined caption describing the generated images (e.g. "a photo of a face" "a photo of a living room") and uses an LLM to hierarchically generate domain-specific attributes. GELDA then uses the LLM to select which VLM from a pre-defined set is most appropriate to annotate each attribute. To demonstrate GELDA's capabilities we present results revealing biases of both text-to-image diffusion models (Stable Diffusion XL) and generative adversarial networks (StyleGAN2). While GELDA is not intended to completely replace human annotators especially for sensitive attribute annotations it can serve as a complementary tool to help humans analyze image generation models in a cheap low-effort and flexible manner. GELDA is available at https://github.com/krishk97/gelda.

Related Material


[pdf]
[bibtex]
@InProceedings{Kabra_2024_CVPR, author = {Kabra, Krish and Lewis, Kathleen M. and Balakrishnan, Guha}, title = {GELDA: A Generative Language Annotation Framework to Reveal Visual Biases in Image Generators}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2024}, pages = {8304-8309} }