RACon: Retrieval-Augmented Simulated Character Locomotion Control

doi:10.1109/ICME57554.2024.10688280

	RACon: Retrieval-Augmented Simulated Character Locomotion Control
	Mu, Yuxuan 1; Zou, Shihao 1; Yin, Kangning2 ; Tian, Zheng2 ; Cheng, Li 1; Zhang, Weinan 3; Wang, Jun 4
	2024
会议录名称	PROCEEDINGS - IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO
ISSN	1945-7871
DOI	10.1109/ICME57554.2024.10688280
摘要	In computer animation, driving a simulated character with lifelike motion is challenging. Current generative models, though able to generalize to diverse motions, often pose challenges to the responsiveness of end-user control. To address these issues, we introduce RACon: Retrieval-Augmented Simulated Character Locomotion Control. Our end-to-end hierarchical reinforcement learning method utilizes a retriever and a motion controller. The retriever searches motion experts from a user-specified database in a task-oriented fashion, which boosts the responsiveness to the user's control. The selected motion experts and the manipulation signal are then transferred to the controller to drive the simulated character. In addition, a retrieval-augmented discriminator is designed to stabilize the training process. Our method surpasses existing techniques in both quality and quantity in locomotion control, as demonstrated in our empirical study. Moreover, by switching extensive databases for retrieval, it can adapt to distinctive motion types at run time. We will release our code upon acceptance. © 2024 IEEE.
会议录编者/会议主办者	IEEE Circuits and Systems (CAS) ; IEEE Communications (ComSoc) ; IEEE Computer (CS) ; IEEE Signal Processing (SPS)
关键词	Augmented reality 'current Computer animation End to end End-users Generative model Locomotion control Physical simulation Reinforcement learnings Retrieval augmented model User control
会议名称	2024 IEEE International Conference on Multimedia and Expo, ICME 2024
会议地点	Niagra Falls, ON, Canada
会议日期	July 15, 2024 - July 19, 2024
URL	查看原文
收录类别	EI
语种	英语
出版者	IEEE Computer Society
EI入藏号	20244317227683
EI主题词	Reinforcement learning
EISSN	1945-788X
EI分类号	1101.2 ; 1107.1
原始文献类型	Conference article (CA)
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/442541
专题	信息科学与技术学院_硕士生创意与艺术学院_PI研究组(P)_田政组
通讯作者	Tian, Zheng
作者单位	1.University of Alberta, Department of Electrical and Computer Engineering, Canada 2.ShanghaiTech University, School of Creativity and Art, China 3.Shanghai Jiao Tong University, Department of Computer Science & Engineering, China 4.University of London College, Department of Computer Science, United Kingdom
通讯作者单位	上海科技大学
推荐引用方式 GB/T 7714	Mu, Yuxuan,Zou, Shihao,Yin, Kangning,et al. RACon: Retrieval-Augmented Simulated Character Locomotion Control[C]//IEEE Circuits and Systems (CAS), IEEE Communications (ComSoc), IEEE Computer (CS), IEEE Signal Processing (SPS):IEEE Computer Society,2024.