Toward multi-target self-organizing pursuit in a partially observable Markov game

doi:10.1016/j.ins.2023.119475

	Toward multi-target self-organizing pursuit in a partially observable Markov game
	Sun, Lijun 1,2; Chang, Yu-Cheng 2; Lyu, Chao 3; Shi, Ye4 ; Shi, Yuhui 1; Lin, Chin-Teng 2
	2023-11
发表期刊	INFORMATION SCIENCES
ISSN	0020-0255
EISSN	1872-6291
卷号	648
发表状态	已发表
DOI	10.1016/j.ins.2023.119475
摘要	The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve the implicit coordination capabilities in search and pursuit. We model a self-organizing system as a partially observable Markov game (POMG) featured by large-scale, decentralization, partial observation, and noncommunication. The proposed distributed algorithm–fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning (MARL) method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that by decomposing the SOP task, FSC2 achieves superior performance compared with other implicit coordination policies fully trained by general MARL algorithms. The scalability of FSC2 is proved that up to 2048 FSC2 agents perform efficient multi-target SOP with almost 100% capture rates. Empirical analyses and ablation studies verify the interpretability, rationality, and effectiveness of component algorithms in FSC2. © 2023 Elsevier Inc.
关键词	Behavioral research Decision making Deep learning Fertilizers Multi agent systems Reinforcement learning Interaction uncertainty Markov games Multi-target pursuits Multi-targets Noncommunication Observation uncertainties Pursuit problems Self organizations Self-organising Uncertainty
URL	查看原文
收录类别	EI ; SCI
语种	英语
资助项目	Shenzhen Fundamental Research Program[JCYJ20200109141235597] ; National Science Foundation of China[61761136008] ; Shenzhen Peacock Plan[KQTD2016112514355531] ; Program for Guangdong Introducing Innovative and Entrepreneurial Teams[2017ZT07X386] ; Australian Research Council (ARC)["DP210101093","DP220100803"]
WOS研究方向	Computer Science
WOS类目	Computer Science, Information Systems
WOS记录号	WOS:001069320000001
出版者	Elsevier Inc.
EI入藏号	20233514638474
EI主题词	Intelligent agents
EI分类号	461.4 Ergonomics and Human Factors Engineering ; 723.4 Artificial Intelligence ; 804 Chemical Products Generally ; 821.2 Agricultural Chemicals ; 912.2 Management ; 971 Social Sciences
原始文献类型	Journal article (JA)
引用统计
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/325772
专题	信息科学与技术学院信息科学与技术学院_PI研究组_石野组
通讯作者	Shi, Yuhui; Lin, Chin-Teng
作者单位	1.Southern Univ Sci & Technol, Dept Comp Sci & Engn, Guangdong Prov Key Lab Brain inspired Intelligent, Shenzhen, Peoples R China 2.Univ Technol Sydney, Fac Engn & Informat Technol, Ctr Artificial Intelligence, CIBCI Lab, Ultimo, Australia 3.Southwest Univ, Coll Comp & Informat Sci, Chongqing, Peoples R China 4.ShanghaiTech Univ, Sch Informat Sci & Technol, Shanghai, Peoples R China
推荐引用方式 GB/T 7714	Sun, Lijun,Chang, Yu-Cheng,Lyu, Chao,et al. Toward multi-target self-organizing pursuit in a partially observable Markov game[J]. INFORMATION SCIENCES,2023,648.
APA	Sun, Lijun,Chang, Yu-Cheng,Lyu, Chao,Shi, Ye,Shi, Yuhui,&Lin, Chin-Teng.(2023).Toward multi-target self-organizing pursuit in a partially observable Markov game.INFORMATION SCIENCES,648.
MLA	Sun, Lijun,et al."Toward multi-target self-organizing pursuit in a partially observable Markov game".INFORMATION SCIENCES 648(2023).