Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses

doi:10.1145/3394171.3413585

	Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses
	Jin, Jing 1; Hou, Junhui 1; Chen, Jie 2; Kwong, Sam 1; Yu, Jingyi3
	2020-10-12
会议录名称	MM 2020 - PROCEEDINGS OF THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA
页码	193-201
DOI	10.1145/3394171.3413585
摘要	This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. To tackle this challenge, we propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation; the other one constructs another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations via the learned attention maps, leading to the final high-resolution LF image. Extensive experiments demonstrate the significant superiority of our approach over state-of-the-art ones. That is, our method not only improves the PSNR by more than 2 dB, but also preserves the LF structure much better. To the best of our knowledge, this is the first end-to-end deep learning method for reconstructing a high-resolution LF image with a hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and also be beneficial to LF data storage and transmission. The code is available at https://github.com/jingjin25/LFhybridSR-Fusion. © 2020 ACM.
会议录编者/会议主办者	ACM SIGMM
关键词	Frequency estimation Learning systems Image reconstruction Cameras Data acquisition Deep learning Optical resolving power Digital storage High frequency HF High resolution High resolution camera Learning methods Learning-based approach Low resolution State of the art Super resolution
会议名称	28th ACM International Conference on Multimedia, MM 2020
会议地点	Virtual, Online, United states
会议日期	October 12, 2020 - October 16, 2020
URL	查看原文
收录类别	EI
语种	英语
出版者	Association for Computing Machinery, Inc
EI入藏号	20212210441315
EI主题词	Textures
EI分类号	461.4 Ergonomics and Human Factors Engineering ; 722.1 Data Storage, Equipment and Techniques ; 723.2 Data Processing and Image Processing ; 741.1 Light/Optics ; 742.2 Photographic Equipment
原始文献类型	Conference article (CA)
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/251849
专题	信息科学与技术学院_PI研究组_虞晶怡组
通讯作者	Hou, Junhui
作者单位	1.City University of Hong Kong, Hong Kong, Hong Kong; 2.Hong Kong Baptist University, Hong Kong, Hong Kong; 3.Shanghai Tech University, Shanghai, China
推荐引用方式 GB/T 7714	Jin, Jing,Hou, Junhui,Chen, Jie,et al. Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses[C]//ACM SIGMM:Association for Computing Machinery, Inc,2020:193-201.