Beyond Myopic Alignment: Lookahead Optimization for Online Class-Incremental Learning

Song Lai, Zhe Zhao, Fei Zhu, Ji Cheng, Xi Lin, Qingfu Zhang, Gaofeng Meng; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 18053-18062

Rehearsal-based methods are the cornerstone of modern online class-incremental learning (OCIL), yet they face a fundamental challenge: the gradient of the current task often conflicts with that of the rehearsal data from the memory buffer, leading to catastrophic forgetting. Recent works have implicitly addressed this by using hypergradients, but the underlying mechanism has remained poorly understood. In this paper, we provide a formal analysis revealing that hypergradients mitigate forgetting by aligning task-specific gradients towards a common meta-objective, thereby reducing their conflict. However, we argue that this conflict-reducing alignment is inherently myopic--it only considers the immediate gradient directions, failing to account for the loss landscape geometry one step ahead. To overcome this limitation, we introduce a novel framework: Lookahead Optimization for Rehearsal (LOR). LOR explores a set of future model states by taking lookahead steps along different directions that balance plasticity and stability and optimizes a first-order Log-Sum-Exp (LSE) surrogate to emphasize the worst-performing sampled lookahead directions. Theoretical analysis from both optimization and statistical perspectives corroborates the robustness of our approach. Extensive experiments on Seq-CIFAR10, Seq-CIFAR100, and Seq-TinyImageNet demonstrate that LOR significantly outperforms state-of-the-art methods, establishing a new and more robust paradigm for rehearsal-based OCIL.