Efficient Wireless Federated Learning Via Low-Rank Gradient Factorization

doi:10.1109/TVT.2024.3506950

	Efficient Wireless Federated Learning Via Low-Rank Gradient Factorization
	Mingzhao Guo1 ; Dongzhu Liu 2; Osvaldo Simeone 3; Dingzhu Wen1
	2024
发表期刊	IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (IF:6.1[JCR-2023],6.5[5-Year])
ISSN	1939-9359
EISSN	1939-9359
卷号	PP 期号:99
发表状态	已发表
DOI	10.1109/TVT.2024.3506950
摘要	This paper presents a novel gradient compression method for federated learning (FL) in wireless systems. The proposed method centers on a low-rank matrix factorization strategy for local gradient compression that is based on one iteration of a distributed Jacobi successive convex approximation (SCA) at each FL round. The low-rank approximation obtained at one round is used as a “warm start” initialization for Jacobi SCA in the next FL round. A new protocol termed over-the-air low-rank compression (Ota-LC) incorporating this gradient compression method with over-the-air computation and error feedback is shown to have lower computation cost and lower communication overhead, while guaranteeing the same inference performance, as compared with existing benchmarks. As an example, when targeting a test accuracy of 70% on the Cifar-10 dataset, Ota-LC reduces total communication costs by at least 33% compared to benchmark schemes.
关键词	Benchmarking Iterative methods Jacobian matrices Compression methods Gradient factorization Low rank compression Low-rank matrices Matrix factorizations Over the airs Over-the-air computation Successive convex approximations Wireless federated learning Wireless systems
URL	查看原文
收录类别	SCI ; EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20244917493410
EI主题词	Matrix factorization
EI分类号	1106.6 ; 1201.2 ; 1201.9 ; 913.3 Quality Assurance and Control
原始文献类型	Article in Press
来源库	IEEE
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/452406
专题	信息科学与技术学院信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_文鼎柱组
作者单位	1.School of Information Science and Technology, ShanghaiTech University, China 2.School of Computing Science, University of Glasgow, U.K. 3.King's Communications, Learning & Information Processing (KCLIP) lab, Centre for Intelligent Information Processing Systems (CIIPS), Department of Engineering, King's College London, London, U.K.
第一作者单位	信息科学与技术学院
第一作者的第一单位	信息科学与技术学院
推荐引用方式 GB/T 7714	Mingzhao Guo,Dongzhu Liu,Osvaldo Simeone,et al. Efficient Wireless Federated Learning Via Low-Rank Gradient Factorization[J]. IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY,2024,PP(99).
APA	Mingzhao Guo,Dongzhu Liu,Osvaldo Simeone,&Dingzhu Wen.(2024).Efficient Wireless Federated Learning Via Low-Rank Gradient Factorization.IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY,PP(99).
MLA	Mingzhao Guo,et al."Efficient Wireless Federated Learning Via Low-Rank Gradient Factorization".IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY PP.99(2024).