On the Importance of Conditioning for Privacy-Preserving Data Augmentation

Julian Lorenz, Katja Ludwig, Valentin Haug, Rainer Lienhart; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 2338-2347

Latent diffusion models can be used as a powerful augmentation method to artificially extend datasets for enhanced training, by swapping parts of the image with generated content which look very different from the originals to the human eye. A recent ECCV oral paper has suggested to use this data augmentation technique for data anonymization. However, we show in this paper that latent diffusion models that are conditioned on modalities like depth maps or edges to guide the diffusion process are not suitable as a privacy-preserving method. We use a contrastive learning approach to train a model that can correctly identify people out of a pool of candidates, with a success rate of over 69% regarding a pool of over 3k persons. Moreover, we demonstrate that anonymization using conditioned diffusion models is susceptible to black box attacks. We attribute the success of the described methods to the conditioning of the latent diffusion model in the anonymization process. The diffusion model is instructed to produce similar edges for the anonymized images. Hence, a model can learn to recognize these patterns for identification.