| |||||||
ShanghaiTech University Knowledge Management System
Alleviating Over-smoothing for Unsupervised Sentence Representation | |
2023-05-04 | |
状态 | 已发表 |
摘要 | Vertical federated learning (FL) is a collaborative machine learning framework that enables devices to learn a global model from the feature-partition datasets without sharing local raw data. However, as the number of the local intermediate outputs is proportional to the training samples, it is critical to develop communication-efficient techniques for wireless vertical FL to support high-dimensional model aggregation with full device participation. In this paper, we propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation by leveraging over-the-air computation (AirComp) and alleviating communication straggler issue with cooperative model aggregation among geographically distributed edge servers. However, the model aggregation error caused by AirComp and quantization errors caused by the limited fronthaul capacity degrade the learning performance for vertical FL. To address these issues, we characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. To improve the learning performance, we establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed. We conduct extensive simulations to demonstrate the effectiveness of the proposed system architecture and optimization framework for vertical FL. |
关键词 | Vertical federated learning cloud radio access network over-the-air computation |
DOI | arXiv:2305.06279 |
相关网址 | 查看原文 |
出处 | Arxiv |
WOS记录号 | PPRN:69038634 |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical& Electronic ; Mathematics |
文献类型 | 预印本 |
条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348058 |
专题 | 信息科学与技术学院 信息科学与技术学院_PI研究组_石远明组 信息科学与技术学院_PI研究组_周勇组 信息科学与技术学院_博士生 信息科学与技术学院_PI研究组_毛奕婕组 |
作者单位 | 1.Shanghai Tech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China 2.Tsinghua Univ, Tsinghua Space Ctr, Beijing Natl Res Ctr Informat Sci & Technol, Beijing 100084, Peoples R China 3.Shanghai Jiao Tong Univ, Dept Elect Engn, Shanghai 201210, Peoples R China |
推荐引用方式 GB/T 7714 | Shi, Yuanming,Xia, Shuhao,Zhou, Yong,et al. Alleviating Over-smoothing for Unsupervised Sentence Representation. 2023. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 |
修改评论
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。