Brain topology improved spiking neural network for efficient reinforcement learning of continuous control

doi:10.3389/fnins.2024.1325062

	Brain topology improved spiking neural network for efficient reinforcement learning of continuous control
	Wang, Yongjian 1,2; Wang, Yansong 3,4; Zhang, Xinhe 1; Du, Jiulin3,5 ; Zhang, Tielin 1,2,3; Xu, Bo 1,2,3,4
	2024-04-16
发表期刊	FRONTIERS IN NEUROSCIENCE (IF:3.2[JCR-2023],4.3[5-Year])
EISSN	1662-453X
卷号	18
发表状态	已发表
DOI	10.3389/fnins.2024.1325062
摘要	The brain topology highly reflects the complex cognitive functions of the biological brain after million-years of evolution. Learning from these biological topologies is a smarter and easier way to achieve brain-like intelligence with features of efficiency, robustness, and flexibility. Here we proposed a brain topology-improved spiking neural network (BT-SNN) for efficient reinforcement learning. First, hundreds of biological topologies are generated and selected as subsets of the Allen mouse brain topology with the help of the Tanimoto hierarchical clustering algorithm, which has been widely used in analyzing key features of the brain connectome. Second, a few biological constraints are used to filter out three key topology candidates, including but not limited to the proportion of node functions (e.g., sensation, memory, and motor types) and network sparsity. Third, the network topology is integrated with the hybrid numerical solver-improved leaky-integrated and fire neurons. Fourth, the algorithm is then tuned with an evolutionary algorithm named adaptive random search instead of backpropagation to guide synaptic modifications without affecting raw key features of the topology. Fifth, under the test of four animal-survival-like RL tasks (i.e., dynamic controlling in Mujoco), the BT-SNN can achieve higher scores than not only counterpart SNN using random topology but also some classical ANNs (i.e., long-short-term memory and multi-layer perception). This result indicates that the research effort of incorporating biological topology and evolutionary learning rules has much in store for the future.
关键词	spiking neural network brain topology hierarchical clustering reinforcement learning neuromorphic computing
URL	查看原文
收录类别	SCI
语种	英语
资助项目	Strategic Priority Research Program of Chinese Academy of Sciences[XDA0370305] ; Beijing Nova Program[20230484369] ; Shanghai Municipal Science and Technology Major Project[2021SHZDZX]
WOS研究方向	Neurosciences & Neurology
WOS类目	Neurosciences
WOS记录号	WOS:001258472700001
出版者	FRONTIERS MEDIA SA
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/398583
专题	生命科学与技术学院生命科学与技术学院_特聘教授组_杜久林组
通讯作者	Du, Jiulin; Zhang, Tielin; Xu, Bo
作者单位	1.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China 2.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing, Peoples R China 3.Chinese Acad Sci, Ctr Excellence Brain Sci & Intelligence Technol, State Key Lab Neurosci, Inst Neurosci, Shanghai, Peoples R China 4.Univ Chinese Acad Sci, Sch Future Technol, Beijing, Peoples R China 5.ShanghaiTech Univ, Sch Life Sci & Technol, Shanghai, Peoples R China
通讯作者单位	生命科学与技术学院
推荐引用方式 GB/T 7714	Wang, Yongjian,Wang, Yansong,Zhang, Xinhe,et al. Brain topology improved spiking neural network for efficient reinforcement learning of continuous control[J]. FRONTIERS IN NEUROSCIENCE,2024,18.
APA	Wang, Yongjian,Wang, Yansong,Zhang, Xinhe,Du, Jiulin,Zhang, Tielin,&Xu, Bo.(2024).Brain topology improved spiking neural network for efficient reinforcement learning of continuous control.FRONTIERS IN NEUROSCIENCE,18.
MLA	Wang, Yongjian,et al."Brain topology improved spiking neural network for efficient reinforcement learning of continuous control".FRONTIERS IN NEUROSCIENCE 18(2024).