Disentangling Light Fields for Super-Resolution and Disparity Estimation

doi:10.1109/TPAMI.2022.3152488

	Disentangling Light Fields for Super-Resolution and Disparity Estimation
	Wang, Yingqian 1; Wang, Longguang 1; Wu, Gaochang 2; Yang, Jungang 1; An, Wei 1; Yu, Jingyi3 ; Guo, Yulan 1
	2023
发表期刊	IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (IF:20.8[JCR-2023],22.2[5-Year])
ISSN	0162-8828
EISSN	1939-3539
卷号	45 期号:1 页码:425-443
发表状态	已发表
DOI	10.1109/TPAMI.2022.3152488
摘要	Light field (LF) cameras record both intensity and directions of light rays, and encode 3D cues into 4D LF images. Recently, many convolutional neural networks (CNNs) have been proposed for various LF image processing tasks. However, it is challenging for CNNs to effectively process LF images since the spatial and angular information are highly inter-twined with varying disparities. In this paper, we propose a generic mechanism to disentangle these coupled information for LF image processing. Specifically, we first design a class of domain-specific convolutions to disentangle LFs from different dimensions, and then leverage these disentangled features by designing task-specific modules. Our disentangling mechanism can well incorporate the LF structure prior and effectively handle 4D LF data. Based on the proposed mechanism, we develop three networks (i.e., DistgSSR, DistgASR and DistgDisp) for spatial super-resolution, angular super-resolution and disparity estimation. Experimental results show that our networks achieve state-of-the-art performance on all these three tasks, which demonstrates the effectiveness, efficiency, and generality of our disentangling mechanism. Project page: https://yingqianwang.github.io/DistgLF/. IEEE
关键词	Cameras Convolution Field emission displays Neural networks Optical resolving power Three dimensional displays Disparity estimations Feature disentangling Field images Image super resolutions Images processing Images reconstruction Light field image processing Light fields Spatial resolution Task analysis Three-dimensional display View synthesis
URL	查看原文
收录类别	EI ; SCI ; SCOPUS
语种	英语
资助项目	National Natural Science Foundation ofChina["U20A20185","61972435","61401474","61921001"]
WOS研究方向	Computer Science ; Engineering
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic
WOS记录号	WOS:000899419900026
出版者	IEEE Computer Society
EI入藏号	20220911725859
EI主题词	Image reconstruction
EI分类号	716.1 Information Theory and Signal Processing ; 722.2 Computer Peripheral Equipment ; 741.1 Light/Optics ; 742.2 Photographic Equipment
原始文献类型	Article in Press
来源库	IEEE
引用统计	正在获取...
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/159573
专题	信息科学与技术学院信息科学与技术学院_PI研究组_虞晶怡组
通讯作者	Yang, Jungang; Guo, Yulan
作者单位	1.Natl Univ Def Technol, Coll Elect Sci & Technol, Changsha 410073, Hunan, Peoples R China 2.Northeastern Univ, State Key Lab Synthet Automat Proc Ind, Shenyang 110819, Peoples R China 3.ShanghaiTech Univ, Sch Informat Sci & Technol, Pudong 201210, Peoples R China
推荐引用方式 GB/T 7714	Wang, Yingqian,Wang, Longguang,Wu, Gaochang,et al. Disentangling Light Fields for Super-Resolution and Disparity Estimation[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(1):425-443.
APA	Wang, Yingqian.,Wang, Longguang.,Wu, Gaochang.,Yang, Jungang.,An, Wei.,...&Guo, Yulan.(2023).Disentangling Light Fields for Super-Resolution and Disparity Estimation.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(1),425-443.
MLA	Wang, Yingqian,et al."Disentangling Light Fields for Super-Resolution and Disparity Estimation".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.1(2023):425-443.