Programmatic Concept Learning for Human Motion Description and Synthesis

Sumith Kulal, Jiayuan Mao, Alex Aiken, Jiajun Wu; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 13843-13852

Abstract


We introduce Programmatic Motion Concepts, a hierarchical motion representation for human actions that captures both low level motion and high level description as motion concepts. This representation enables human motion description, interactive editing, and controlled synthesis of novel video sequences within a single framework. We present an architecture that learns this concept representation from paired video and action sequences in a semi-supervised manner. The compactness of our representation also allows us to present a low-resource training recipe for data-efficient learning. By outperforming established baselines, especially in small data regime, we demonstrate the efficiency and effectiveness of our framework for multiple applications.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Kulal_2022_CVPR, author = {Kulal, Sumith and Mao, Jiayuan and Aiken, Alex and Wu, Jiajun}, title = {Programmatic Concept Learning for Human Motion Description and Synthesis}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {13843-13852} }