Layout-Guided Novel View Synthesis from a Single Indoor Panorama

doi:10.1109/CVPR46437.2021.01617

	Layout-Guided Novel View Synthesis from a Single Indoor Panorama
	Xu, Jiale1 ; Zheng, Jia 2; Xu, Yanyu 3; Tang, Rui 2; Gao, Shenghua1,4
	2021
会议录名称	2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021
ISSN	1063-6919
发表状态	已发表
DOI	10.1109/CVPR46437.2021.01617
摘要	Existing view synthesis methods mainly focus on the perspective images and have shown promising results. However, due to the limited field-of-view of the pinhole camera, the performance quickly degrades when large camera movements are adopted. In this paper, we make the first attempt to generate novel views from a single indoor panorama and take the large camera translations into consideration. To tackle this challenging problem, we first use Convolutional Neural Networks (CNNs) to extract the deep features and estimate the depth map from the source-view image. Then, we leverage the room layout prior, a strong structural constraint of the indoor scene, to guide the generation of target views. More concretely, we estimate the room layout in the source view and transform it into the target viewpoint as guidance. Meanwhile, we also constrain the room layout of the generated target-view images to enforce geometric consistency. To validate the effectiveness of our method, we further build a large-scale photo-realistic dataset containing both small and large camera translations. The experimental results on our challenging dataset demonstrate that our method achieves state-of-the-art performance. The project page is at https://github.com/bluestyle97/PNVS.
会议名称	IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
出版地	10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
会议地点	null,null,ELECTR NETWORK
会议日期	JUN 19-25, 2021
URL	查看原文
收录类别	CPCI-S ; EI ; CPCI
语种	英语
资助项目	National Key R&D Program of China[2018AAA0100704] ; National Natural Science Foundation of China[61932020] ; Science and Technology Commission of Shanghai Municipality[20ZR1436000]
WOS研究方向	Computer Science ; Imaging Science & Photographic Technology
WOS类目	Computer Science, Artificial Intelligence ; Imaging Science & Photographic Technology
WOS记录号	WOS:000742075006064
出版者	IEEE COMPUTER SOC
来源库	IEEE
引用统计	被引频次：14[WOS] [WOS记录] [WOS相关记录]
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/153571
专题	信息科学与技术学院_博士生信息科学与技术学院_PI研究组_高盛华组
通讯作者	Gao, Shenghua
作者单位	1.ShanghaiTech Univ, Shanghai, Peoples R China 2.Manycore, KooLab, Hangzhou, Peoples R China 3.ASTAR, Inst High Performance Comp, Singapore, Singapore 4.Shanghai Engn Res Ctr Intelligent Vis & Imaging, Shanghai, Peoples R China
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Xu, Jiale,Zheng, Jia,Xu, Yanyu,et al. Layout-Guided Novel View Synthesis from a Single Indoor Panorama[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2021.