Trade-off on Sim2Real Learning: Real-world Learning Faster than Simulations

doi:10.1109/ICCAR55106.2022.9782656

	Trade-off on Sim2Real Learning: Real-world Learning Faster than Simulations
	Jingyi Huang1 ; Yizheng Zhang1 ; Fabio Giardina 2; Andre Rosendo1
	2022
会议录名称	PROCEEDINGS 2022 8TH INTERNATIONAL CONFERENCE ON CONTROL, AUTOMATION AND ROBOTICS (ICCAR)
ISSN	2251-2446
页码	95-100
发表状态	已发表
DOI	10.1109/ICCAR55106.2022.9782656
摘要	Deep Reinforcement Learning (DRL) experiments are commonly performed in simulated environments due to the tremendous training sample demands from deep neural networks. In contrast, model-based Bayesian Learning allows a robot to learn good policies within a few trials in the real world. Although it takes fewer iterations, Bayesian methods pay a relatively higher computational cost per trial, and the advantage of such methods is strongly tied to dimensionality and noise. In here, we compare a Deep Bayesian Learning algorithm with a model-free DRL algorithm while analyzing our results collected from both simulations and real-world experiments. While considering Sim and Real learning, our experiments show that the sample-efficient Deep Bayesian RL performance is better than DRL even when computation time (as opposed to number of iterations) is taken in consideration. Additionally, the difference in computation time between Deep Bayesian RL performed in simulation and in experiments point to a viable path to traverse the reality gap. We also show that a mix between Sim and Real does not outperform a purely Real approach, pointing to the possibility that reality can provide the best prior knowledge to a Bayesian Learning. Roboticists design and build robots every day, and our results show that a higher learning efficiency in the real-world will shorten the time between design and deployment by skipping simulations.
关键词	deep reinforcement learning robotics decision making
会议名称	INTERNATIONAL CONFERENCE ON CONTROL, AUTOMATION AND ROBOTICS (ICCAR)
会议地点	Virtual, Online
会议日期	April 8, 2022 - April 10, 2022
URL	查看原文
收录类别	EI
语种	英语
EI入藏号	/
来源库	IEEE
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/183468
专题	信息科学与技术学院信息科学与技术学院_PI研究组_ANDRE LUIS MACEDO ROSENDO SILVA组信息科学与技术学院_硕士生
作者单位	1.School of Information Science and Technology ShanghaiTech University, Shanghai, China 2.John A. Paulson School of Engineering and Applied Sciences Harvard University, Cambridge, Massachusetts, United States
第一作者单位	信息科学与技术学院
第一作者的第一单位	信息科学与技术学院
推荐引用方式 GB/T 7714	Jingyi Huang,Yizheng Zhang,Fabio Giardina,et al. Trade-off on Sim2Real Learning: Real-world Learning Faster than Simulations[C],2022:95-100.