FinPercep-RM: A Fine-grained Reward Model and Co-evolutionary Curriculum for RL-based Real-world Super-Resolution

Yidi Liu, Zihao Fan, Jie Huang, Jie Xiao, Dong Li, Wenlong Zhang, LEI BAI, Xueyang Fu, Zheng-jun Zha; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 4839-4849

Inspired by the success of Reinforcement Learning with Human Feedback (RLHF) in image generation, recent work has adapted reward-based learning to image super-resolution (ISR) by using Image Quality Assessment (IQA) models as rewards. However, existing IQA models typically output only a single global score and are insensitive to local, fine-grained distortions, allowing perceptually undesirable artifacts to obtain spuriously high rewards and leading to reward hacking. To address this issue, we propose FinPercep-RM, a fine-grained perceptual reward model built on an encoder-decoder architecture that predicts both a global quality score and a Perceptual Degradation Map for spatially localizing and quantifying local defects. We further introduce FGR-30k, a dataset containing diverse and subtle distortions produced by real-world super-resolution models, to train the reward model. While FinPercep-RM provides stronger supervision, its increased complexity also makes generator policy learning unstable. We therefore develop a Co-evolutionary Curriculum Learning (CCL) strategy, in which the reward model and the ISR model evolve synchronously: the reward signal progressively increases in complexity, while the ISR model starts with simple global supervision for fast convergence and gradually transitions to fine-grained rewards. This easy-to-hard design stabilizes training and suppresses reward hacking. Extensive experiments across multiple ISR models demonstrate improvements in both global quality and local realism. Code will be available at https://github.com/lyd-2022/FinPercep-RM.