Quantifying Generative Stability: Mode Collapse Entropy Score for Mode Diversity Evaluation

Jens Duym, José Antonio Oramas Mogrovejo, Ali Anwar; Proceedings of the Winter Conference on Applications of Computer Vision (WACV) Workshops, 2025, pp. 187-196

Generative models have made a lot of progress in recent years leading to a plethora of evaluation metrics to quantify the quality of the generated images. One critical aspect of quality is the diversity of the plausible images generated by the model which is frequently assessed through metrics designed to measure mode collapse. However existing metrics for evaluating mode collapse in generative models are often insufficient as they rely on simplified datasets such as synthetic data or stacked MNIST which make them inadequate for capturing this phenomenon when generalized to more complex real-world scenarios. This demonstrates the need for a novel evaluation method to accurately assess and compare generative models. This paper addresses this need with the Mode Collapse Entropy (MCE) score. Through experiments on both synthetic and real-world datasets we demonstrate that the MCE score provides accurate and interpretable measurements of mode collapse.