Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control

doi:10.1109/TVCG.2019.2938520

	Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control
	Wang, Zhiyong 1,3; Chai, Jinxiang 2; Xia, Shihong 1,3
刊名	IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
	2021
卷号	27 期号:1 页码:14-28
关键词	Hidden Markov models Data models Training Generators Mathematical model Recurrent neural networks Animation Deep learning adversarial training human motion modeling synthesis and control
ISSN号	1077-2626
DOI	10.1109/TVCG.2019.2938520
英文摘要	This paper introduces a new generative deep learning network for human motion synthesis and control. Our key idea is to combine recurrent neural networks (RNNs) and adversarial training for human motion modeling. We first describe an efficient method for training an RNN model from prerecorded motion data. We implement RNNs with long short-term memory (LSTM) cells because they are capable of addressing the nonlinear dynamics and long term temporal dependencies present in human motions. Next, we train a refiner network using an adversarial loss, similar to generative adversarial networks (GANs), such that refined motion sequences are indistinguishable from real mocap data using a discriminative network. The resulting model is appealing for motion synthesis and control because it is compact, contact-aware, and can generate an infinite number of naturally looking motions with infinite lengths. Our experiments show that motions generated by our deep learning model are always highly realistic and comparable to high-quality motion capture data. We demonstrate the power and effectiveness of our models by exploring a variety of applications, ranging from random motion synthesis, online/offline motion control, and motion filtering. We show the superiority of our generative model by comparison against baseline models.
资助项目	National Natural Science Foundation of China[61772499] ; Natural Science Foundation of Beijing Municipality[L182052]
WOS研究方向	Computer Science
语种	英语
出版者	IEEE COMPUTER SOC
WOS记录号	WOS:000594242000002
内容类型	期刊论文
源URL	[http://119.78.100.204/handle/2XEOYT63/16502]
专题	中国科学院计算技术研究所
通讯作者	Chai, Jinxiang; Xia, Shihong
作者单位	1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China 2.Texas A&M Univ, Uvalde, TX 78801 USA 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式 GB/T 7714	Wang, Zhiyong,Chai, Jinxiang,Xia, Shihong. Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2021,27(1):14-28.
APA	Wang, Zhiyong,Chai, Jinxiang,&Xia, Shihong.(2021).Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,27(1),14-28.
MLA	Wang, Zhiyong,et al."Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Control".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 27.1(2021):14-28.