MoE-AGIQA: Mixture-of-Experts Boosted Visual Perception-Driven and Semantic-Aware Quality Assessment for AI-Generated Images

Junfeng Yang, Jing Fu, Wei Zhang, Wenzhi Cao, Limei Liu, Han Peng; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 6395-6404

Recently there has been a surge of interest in AI-Generated Image Quality Assessment (AGIQA). Unlike images in common image quality assessment tasks AI-generated images may suffer from some unique degradations. To this end we propose a novel mixture-of-experts boosted visual perception-driven and semantic-aware quality assessment for AI-generated images (MoE-AGIQA). Firstly we design a visual degradation-aware network to ascertain perceptual rules by emulating human perception of visual degradation. To enhance the diversity of visual degradation-aware features we additionally devise a prior knowledge injection module which is pre-trained on specific natural images. Secondly we devise a semantic-aware network to assess the inconsistency between input text prompts and AI-generated images and further detect potential semantic problems. Thirdly we propose to conduct cross-attention on visual degradation-aware and semantic-aware features so that we can obtain comprehensive quality-aware features and the inherent correlation between these features. Finally we propose a mixture-of-experts module involving multiple experts working collaboratively. Each expert is responsible for a specific set of features and outputs a corresponding prediction score. The mixture of multiple experts will ultimately yield a holistic perceptual quality score. Experimental results on benchmark AGIQA datasets and the NTIRE 2024 Quality Assessment for AI-Generated Content - Track 1 Image Challenge demonstrate our superior performance. The source code is available at https://github.com/37s/MoE-AGIQA.