Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects

doi:10.1111/cgf.14689

	Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
	Wang, Ziyu1,4 ; Deng, Yu 2,3,4; Yang, Jiaolong 3; Yu, Jingyi1 ; Tong, Xin 3
	2022-10
发表期刊	COMPUTER GRAPHICS FORUM (IF:2.7[JCR-2023],2.9[5-Year])
ISSN	0167-7055
EISSN	1467-8659
卷号	41 期号:7 页码:431-442
发表状态	已发表
DOI	10.1111/cgf.14689
摘要	3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D-aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high-quality texture and shape editing results. © 2022 The Author(s) Computer Graphics Forum © 2022 The Eurographics Association and John Wiley & Sons Ltd. Published by John Wiley & Sons Ltd.
关键词	Image reconstruction Rendering (computer graphics) Textures 2D images CCS concept Computing methodologies Generative model Image manipulation Images synthesis Monocular image Performance Shape Modelling • computing methodology → rendering
URL	查看原文
收录类别	EI ; SCI ; CPCI-S
语种	英语
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:001008797000039
出版者	John Wiley and Sons Inc
EI入藏号	20231313801810
EI主题词	Topology
EI分类号	723.2 Data Processing and Image Processing - 723.5 Computer Applications - 921.4 Combinatorial Mathematics, Includes Graph Theory, Set Theory
原始文献类型	Journal article (JA)
引用统计	正在获取...
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/294836
专题	信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_博士生
通讯作者	Wang, Ziyu
作者单位	1.ShanghaiTech Univ, Shanghai, Peoples R China 2.Tsinghua Univ, Beijing, Peoples R China 3.Microsoft Res Asia, Beijing, Peoples R China 4.MSRA, Beijing, Peoples R China
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Wang, Ziyu,Deng, Yu,Yang, Jiaolong,et al. Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects[J]. COMPUTER GRAPHICS FORUM,2022,41(7):431-442.
APA	Wang, Ziyu,Deng, Yu,Yang, Jiaolong,Yu, Jingyi,&Tong, Xin.(2022).Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects.COMPUTER GRAPHICS FORUM,41(7),431-442.
MLA	Wang, Ziyu,et al."Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects".COMPUTER GRAPHICS FORUM 41.7(2022):431-442.