Transformer Based Motion In-Betweening

Pavithra Sridhar, Aananth V, Madhav Aggarwal, R Leela Velusamy; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2022, pp. 289-302

In-Betweening is the process of drawing transition frames between temporally-sparse keyframes to create a smooth animation sequence. This work presents a novel transformer based in betweening technique that serves as a tool for 3D animators. We first show that this problem can be represented as a sequence to sequence problem and introduce TweenTransformers - a model that synthesizes high-quality animations using temporally-sparse keyframes as input constraints. We evaluate the model's performance via two complementary methods-quantitative evaluation and qualitative evaluation. The model is compared quantitatively with the state-of-the-art models using LaFAN1, a high-quality animation dataset. Mean-squared metrics like L2P, L2Q, and NPSS are used for evaluation. Qualitatively, we provide two straightforward methods to assess the model's output. First, we implement a custom ThreeJs-based motion visualizer to render the ground truth, input, and output sequences side by side for comparison. The visualizer renders custom sequences by specifying skeletal positions at temporally-sparse keyframes in JSON format. Second, we build a motion generator to generate custom motion sequences using the model. Code can be found in https://github.com/Pavi114/motion-completion-using-transformers