Automated Contrastive Learning Strategy Search for Time Series

doi:10.1145/3627673.3680086

	Automated Contrastive Learning Strategy Search for Time Series
	Jing, Baoyu 1; Wang, Yansen 2; Sui, Guoxin 2; Hong, Jing 3; He, Jingrui 1; Yang, Yuqing 2; Li, Dongsheng 2; Ren, Kan4
	2024-10-21
会议录名称	INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, PROCEEDINGS
ISSN	2155-0751
页码	4612-4620
DOI	10.1145/3627673.3680086
摘要	In recent years, Contrastive Learning (CL) has become a predominant representation learning paradigm for time series. Most existing methods manually build specific CL Strategies (CLS) by human heuristics for certain datasets and tasks. However, manually developing CLS usually requires excessive prior knowledge about the data, and massive experiments to determine the detailed CL configurations. In this paper, we present an Automated Machine Learning (AutoML) practice at Microsoft, which automatically learns CLS for time series datasets and tasks, namely Automated Contrastive Learning (AutoCL). We first construct a principled search space of size over 3 × 1012, covering data augmentation, embedding transformation, contrastive pair construction, and contrastive losses. Further, we introduce an efficient reinforcement learning algorithm, which optimizes CLS from the performance on the validation tasks, to obtain effective CLS within the space. Experimental results on various real-world datasets demonstrate that AutoCL could automatically find the suitable CLS for the given dataset and task. From the candidate CLS found by AutoCL on several public datasets/tasks, we compose a transferable Generally Good Strategy (GGS), which has a strong performance for other datasets. We also provide empirical analysis as a guide for the future design of CLS. © 2024 ACM.
会议录编者/会议主办者	ACM SIGIR ; ACM SIGWEB
关键词	Adversarial machine learning Federated learning Reinforcement learning Automated machine learning Automated machines Learn+ Learning paradigms Learning strategy Machine-learning MicroSoft Performance Prior-knowledge Times series
会议名称	33rd ACM International Conference on Information and Knowledge Management, CIKM 2024
会议地点	Boise, ID, United states
会议日期	October 21, 2024 - October 25, 2024
URL	查看原文
收录类别	EI
语种	英语
出版者	Association for Computing Machinery
EI入藏号	20244817429678
EI主题词	Contrastive Learning
EI分类号	1101.2
原始文献类型	Conference article (CA)
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/455183
专题	信息科学与技术学院_PI研究组_任侃组
通讯作者	Ren, Kan
作者单位	1.University of Illinois at Urbana-Champaign, Champaign; IL, United States; 2.Microsoft Research Asia, Shanghai, China; 3.Ruijin Hospital, Shanghai, China; 4.ShanghaiTech University, Shanghai, China
通讯作者单位	上海科技大学
推荐引用方式 GB/T 7714	Jing, Baoyu,Wang, Yansen,Sui, Guoxin,et al. Automated Contrastive Learning Strategy Search for Time Series[C]//ACM SIGIR, ACM SIGWEB:Association for Computing Machinery,2024:4612-4620.