Text-Guided Patch Scoring and Local Distortion Guidance for Image Quality Assessment

Juyong Park, Jihun Song, Gyewan Kim, Yoonsuk Hyun; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025, pp. 790-799

Image Quality Assessment (IQA) aims to provide objective quality scores of images by imitating the Human Visual System (HVS). Several IQA studies have produced promising results with the patch-wise prediction, which applies the weighted averaging of patch scores to predict image quality scores. However, these studies have implemented the patch-wise prediction using only visual information. With the success of Vision-Language Models (VLMs), we aim to develop a patch-wise prediction specialized for VLMs. To achieve this, we propose Text-Guided Patch Scoring via Multi-Level Features of Vision-Language Models for NR-IQA (TeMu-IQA). Specifically, our model aggregates multi-level features from the image encoder of the frozen VLM and leverages the patch-wise prediction with text-guided patch scoring (TPS). TeMu-IQA achieves state-of-the-art performance on various IQA datasets, even with a few trainable parameters, and exhibits consistency by maintaining superior performance across diverse VLMs. Moreover, we propose Local Distortion Guidance (LDG), a novel methodology that addresses the over-localized problem of the patch-wise prediction where the image's overall structural characteristic is not sufficiently considered. By generating a locally distorted image and guiding it to receive a lower quality score than the original, LDG strengthens the model's ability to reflect the image's logical coherence in its quality assessment.