GNeRF: GAN-based Neural Radiance Field without Posed Camera

doi:10.1109/ICCV48922.2021.00629

	GNeRF: GAN-based Neural Radiance Field without Posed Camera
	Meng, Quan1 ; Chen, Anpei1 ; Luo, Haimin1 ; Wu, Mine1 ; Su, Hao 2; Xu, Lan1 ; He, Xuming1 ; Yu, Jingyi1
	2021-10-17
会议录名称	IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)
ISSN	2380-7504
发表状态	已发表
DOI	10.1109/ICCV48922.2021.00629
摘要	We introduce GNeRF, a framework to marry Generative Adversarial Networks (GAN) with Neural Radiance Field (NeRF) reconstruction for the complex scenarios with unknown and even randomly initialized camera poses. Recent NeRF-based advances have gained popularity for remarkable realistic novel view synthesis. However, most of them heavily rely on accurate camera poses estimation, while few recent methods can only optimize the unknown camera poses in roughly forward-facing scenes with relatively short camera trajectories and require rough camera poses initialization. Differently, our GNeRF only utilizes randomly initialized poses for complex outside-in scenarios. We propose a novel two-phases end-to-end framework. The first phase takes the use of GANs into the new realm for optimizing coarse camera poses and radiance fields jointly, while the second phase refines them with additional photometric loss. We overcome local minima using a hybrid and iterative optimization scheme. Extensive experiments on a variety of synthetic and natural scenes demonstrate the effectiveness of GNeRF. More impressively, our approach outperforms the baselines favorably in those scenes with repeated patterns or even low textures that are regarded as extremely challenging before.
会议名称	18th IEEE/CVF International Conference on Computer Vision (ICCV)
出版地	345 E 47TH ST, NEW YORK, NY 10017 USA
会议地点	null,null,ELECTR NETWORK
会议日期	OCT 11-17, 2021
URL	查看原文
收录类别	EI ; CPCI ; CPCI-S
语种	英语
资助项目	NSFC[61976138,61977047] ; National Key Research and Development Program[2018YFB2100500] ; STCSM[2015F0203-000-06] ; SHMEC[2019-01-07-00-01-E00003]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/183381
专题	信息科学与技术学院_博士生信息科学与技术学院_PI研究组_何旭明组信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_许岚组
通讯作者	Meng, Quan
作者单位	1.ShanghaiTech University 2.UCSD
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Meng, Quan,Chen, Anpei,Luo, Haimin,et al. GNeRF: GAN-based Neural Radiance Field without Posed Camera[C]. 345 E 47TH ST, NEW YORK, NY 10017 USA,2021.