An Empirical Study on Google Research Football Multi-agent Scenarios

doi:10.1007/s11633-023-1426-8

	An Empirical Study on Google Research Football Multi-agent Scenarios
	Song, Yan 1; Jiang, He 2; Tian, Zheng3 ; Zhang, Haifeng 1; Zhang, Yingping 4; Zhu, Jiangcheng 4; Dai, Zonghong 4; Zhang, Weinan 5; Wang, Jun 6
	2024
发表期刊	MACHINE INTELLIGENCE RESEARCH
ISSN	2731-538X
EISSN	2731-5398
卷号	21 期号:3 页码:549-570
DOI	10.1007/s11633-023-1426-8
摘要	Few multi-agent reinforcement learning (MARL) researches on Google research football (GRF) focus on the 11-vs-11 multi-agent full-game scenario and to the best of our knowledge, no open benchmark on this scenario has been released to the public. In this work, we fill the gap by providing a population-based MARL training pipeline and hyperparameter settings on multi-agent football scenario that outperforms the bot with difficulty 1.0 from scratch within 2 million steps. Our experiments serve as a reference for the expected performance of independent proximal policy optimization (IPPO), a state-of-the-art multi-agent reinforcement learning algorithm where each agent tries to maximize its own policy independently across various training configurations. Meanwhile, we release our training framework Light-MALib which extends the MALib codebase by distributed and asynchronous implementation with additional analytical tools for football games. Finally, we provide guidance for building strong football AI with population-based training and release diverse pretrained policies for benchmarking. The goal is to provide the community with a head start for whoever experiment their works on GRF and a simple-to-use population-based training framework for further improving their agents through self-play. The implementation is available at https://github.com/Shanghai-Digital-Brain-Laboratory/DB-Football . © 2024, Institute of Automation, Chinese Academy of Sciences and Springer-Verlag GmbH Germany, part of Springer Nature.
关键词	Fertilizers HTTP Learning algorithms Learning systems Multi agent systems Reinforcement learning Distributed reinforcement learning system Empirical studies Google+ Multi agent Multi-agent reinforcement learning Population-based training Reinforcement learning systems Reward shaping Training framework
URL	查看原文
收录类别	EI
语种	英语
出版者	Chinese Academy of Sciences
EI入藏号	20240315385682
EI主题词	Game theory
EI分类号	723.4 Artificial Intelligence ; 723.4.2 Machine Learning ; 804 Chemical Products Generally ; 821.2 Agricultural Chemicals ; 922.1 Probability Theory
原始文献类型	Article in Press
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/349763
专题	创意与艺术学院_PI研究组(P)_田政组
通讯作者	Tian, Zheng; Zhang, Weinan
作者单位	1.Institute of Automation, Chinese Academy of Sciences, Beijing; 100190, China 2.Digital Brain Lab, Shanghai; 200001, China 3.ShanghaiTech University, Shanghai; 200001, China 4.Huawei Cloud, Guiyang; 550003, China 5.Shanghai Jiao Tong University, Shanghai; 200001, China 6.University College London, London; WC1E 6PT, United Kingdom
通讯作者单位	上海科技大学
推荐引用方式 GB/T 7714	Song, Yan,Jiang, He,Tian, Zheng,et al. An Empirical Study on Google Research Football Multi-agent Scenarios[J]. MACHINE INTELLIGENCE RESEARCH,2024,21(3):549-570.
APA	Song, Yan.,Jiang, He.,Tian, Zheng.,Zhang, Haifeng.,Zhang, Yingping.,...&Wang, Jun.(2024).An Empirical Study on Google Research Football Multi-agent Scenarios.MACHINE INTELLIGENCE RESEARCH,21(3),549-570.
MLA	Song, Yan,et al."An Empirical Study on Google Research Football Multi-agent Scenarios".MACHINE INTELLIGENCE RESEARCH 21.3(2024):549-570.