Corgi: Cached Memory Guided Video Generation

Xindi Wu, Uriel Singer, Zhaojiang Lin, Andrea Madotto, Xide Xia, Yifan Xu, Paul Crook, Xin Luna Dong, Seungwhan Moon; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 4585-4594

Text-to-Video generation has achieved remarkable progress with the rise of diffusion models. In this work we introduce Cached Memory-Guided Video Generation (Corgi) aiming to generate multi-scene videos with arbitrary number of video clips conditioned on input images and instruction prompts. This is a challenging task as traditional T2V methods often struggle to maintain the quality of longer videos due to the difficulties in preserving visual context from earlier scenes. We address this by introducing a cached memory mechanism that stores the key frames. Our multi-scene video generation process is explicitly conditioned on the cached memories to avoid forgetting the visual appearance of target subjects. Corgi shows significant improvement in multi-scene video generation compared to the prior art with up to 59.2% in long-term consistency and 7.6% in diversity.