Revisiting Event-based Video Frame Interpolation

doi:10.1109/IROS55552.2023.10341804

	Revisiting Event-based Video Frame Interpolation
	Jiaben Chen 1; Yichen Zhu2 ; Dongze Lian 3; Jiaqi Yang2 ; Yifu Wang2 ; Renrui Zhang 4; Xinhang Liu2 ; Shenhan Qian 5; Laurent Kneip2 ; Shenghua Gao2
	2023
会议录名称	PROCEEDINGS OF IEEE/RSJ INTERNATIONAL CONFERENCE ON INTELLIGENT ROBOTS AND SYSTEMS
ISSN	2153-0858
页码	1292-1299
发表状态	已发表
DOI	10.1109/IROS55552.2023.10341804
摘要	Dynamic vision sensors or event cameras provide rich complementary information for video frame interpolation. Existing state-of-the-art methods follow the paradigm of combining both synthesis-based and warping networks. However, few of those methods fully respect the intrinsic characteristics of events streams. Given that event cameras only encode intensity changes and polarity rather than color intensities, estimating optical flow from events is arguably more difficult than from RGB information. We therefore propose to incorporate RGB information in an event-guided optical flow refinement strategy. Moreover, in light of the quasi-continuous nature of the time signals provided by event cameras, we propose a divide-and-conquer strategy in which event-based intermediate frame synthesis happens incrementally in multiple simplified stages rather than in a single, long stage. Extensive experiments on both synthetic and real-world datasets show that these modifications lead to more reliable and realistic intermediate frame results than previous video frame interpolation methods. Our findings underline that a careful consideration of event characteristics such as high temporal density and elevated noise benefits interpolation accuracy. © 2023 IEEE.
关键词	Cameras Computer vision Interpolation Color intensity Dynamic vision sensors Event streams Event-based Frame interpolation Intensity change Intrinsic characteristics State-of-the-art methods Video frame Warpings
会议名称	2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2023
会议地点	Detroit, MI, United states
会议日期	October 1, 2023 - October 5, 2023
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20240315412493
EI主题词	Optical flows
EISSN	2153-0866
EI分类号	723.5 Computer Applications ; 741.1 Light/Optics ; 741.2 Vision ; 742.2 Photographic Equipment ; 921.6 Numerical Methods
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/346068
专题	信息科学与技术学院_博士生信息科学与技术学院_PI研究组_高盛华组信息科学与技术学院_PI研究组_Laurent Kneip组信息科学与技术学院_硕士生信息科学与技术学院_本科生
共同第一作者	Yichen Zhu
通讯作者	Shenghua Gao
作者单位	1.UC San Diego 2.ShanghaiTech University 3.National University of Singapore 4.Shanghai AI Laboratory 5.Technical University of Munich
通讯作者单位	上海科技大学
推荐引用方式 GB/T 7714	Jiaben Chen,Yichen Zhu,Dongze Lian,et al. Revisiting Event-based Video Frame Interpolation[C]:Institute of Electrical and Electronics Engineers Inc.,2023:1292-1299.