消息
×
loading..
Alleviating Over-smoothing for Unsupervised Sentence Representation
2023-05-04
状态已发表
摘要

Vertical federated learning (FL) is a collaborative machine learning framework that enables devices to learn a global model from the feature-partition datasets without sharing local raw data. However, as the number of the local intermediate outputs is proportional to the training samples, it is critical to develop communication-efficient techniques for wireless vertical FL to support high-dimensional model aggregation with full device participation. In this paper, we propose a novel cloud radio access network (Cloud-RAN) based vertical FL system to enable fast and accurate model aggregation by leveraging over-the-air computation (AirComp) and alleviating communication straggler issue with cooperative model aggregation among geographically distributed edge servers. However, the model aggregation error caused by AirComp and quantization errors caused by the limited fronthaul capacity degrade the learning performance for vertical FL. To address these issues, we characterize the convergence behavior of the vertical FL algorithm considering both uplink and downlink transmissions. To improve the learning performance, we establish a system optimization framework by joint transceiver and fronthaul quantization design, for which successive convex approximation and alternate convex search based system optimization algorithms are developed. We conduct extensive simulations to demonstrate the effectiveness of the proposed system architecture and optimization framework for vertical FL.

关键词Vertical federated learning cloud radio access network over-the-air computation
DOIarXiv:2305.06279
相关网址查看原文
出处Arxiv
WOS记录号PPRN:69038634
WOS类目Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Engineering, Electrical& Electronic ; Mathematics
文献类型预印本
条目标识符https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348058
专题信息科学与技术学院
信息科学与技术学院_PI研究组_石远明组
信息科学与技术学院_PI研究组_周勇组
信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_毛奕婕组
作者单位
1.Shanghai Tech Univ, Sch Informat Sci & Technol, Shanghai 201210, Peoples R China
2.Tsinghua Univ, Tsinghua Space Ctr, Beijing Natl Res Ctr Informat Sci & Technol, Beijing 100084, Peoples R China
3.Shanghai Jiao Tong Univ, Dept Elect Engn, Shanghai 201210, Peoples R China
推荐引用方式
GB/T 7714
Shi, Yuanming,Xia, Shuhao,Zhou, Yong,et al. Alleviating Over-smoothing for Unsupervised Sentence Representation. 2023.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
个性服务
查看访问统计
谷歌学术
谷歌学术中相似的文章
[Shi, Yuanming]的文章
[Xia, Shuhao]的文章
[Zhou, Yong]的文章
百度学术
百度学术中相似的文章
[Shi, Yuanming]的文章
[Xia, Shuhao]的文章
[Zhou, Yong]的文章
必应学术
必应学术中相似的文章
[Shi, Yuanming]的文章
[Xia, Shuhao]的文章
[Zhou, Yong]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。