Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild

doi:10.1109/CVPR46437.2021.01519

	Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild
	Yin, Zhaoyuan 1; Zheng, Jia 2; Luo, Weixin 3; Qian, Shenhan4 ; Zhang, Hanling 5; Gao, Shenghua4,6
	2021
会议录名称	2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021
ISSN	1063-6919
发表状态	已发表
DOI	10.1109/CVPR46437.2021.01519
摘要	This paper proposes a framework for the interactive video object segmentation (VOS) in the wild where users can choose some frames for annotations iteratively. Then, based on the user annotations, a segmentation algorithm refines the masks. The previous interactive VOS paradigm selects the frame with some worst evaluation metric, and the ground truth is required for calculating the evaluation metric, which is impractical in the testing phase. In contrast, in this paper, we advocate that the frame with the worst evaluation metric may not be exactly the most valuable frame that leads to the most performance improvement across the video. Thus, we formulate the frame selection problem in the interactive VOS as a Markov Decision Process, where an agent is learned to recommend the frame under a deep reinforcement learning framework. The learned agent can automatically determine the most valuable frame, making the interactive setting more practical in the wild. Experimental results on the public datasets show the effectiveness of our learned agent without any changes to the underlying VOS algorithms.
会议名称	IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
出版地	10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
会议地点	null,null,ELECTR NETWORK
会议日期	JUN 19-25, 2021
URL	查看原文
收录类别	CPCI-S ; EI ; CPCI
语种	英语
资助项目	Special Funds for the Construction of Innovative Provinces in Hunan[2019NK2022] ; NSFC[61672222,61932020] ; National Key R&D Program of China[2018AAA0100704] ; Science and Technology Commission of Shanghai Municipality[20ZR1436000]
WOS研究方向	Computer Science ; Imaging Science & Photographic Technology
WOS类目	Computer Science, Artificial Intelligence ; Imaging Science & Photographic Technology
WOS记录号	WOS:000742075005065
出版者	IEEE COMPUTER SOC
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/153572
专题	信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_高盛华组
通讯作者	Zhang, Hanling
作者单位	1.Hunan Univ, Coll Comp Sci & Elect Engn, Changsha, Peoples R China 2.Manycore, KooLab, Rancho Cordova, CA USA 3.Meituan Grp, Beijing, Peoples R China 4.ShanghaiTech Univ, Shanghai, Peoples R China 5.Hunan Univ, Sch Design, Changsha, Peoples R China 6.Shanghai Engn Res Ctr Intelligent Vis & Imaging, Shanghai, Peoples R China
推荐引用方式 GB/T 7714	Yin, Zhaoyuan,Zheng, Jia,Luo, Weixin,et al. Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2021.