Relightable Neural Human Assets from Multi-view Gradient Illuminations

doi:10.1109/CVPR52729.2023.00420

	Relightable Neural Human Assets from Multi-view Gradient Illuminations
	Taotao Zhou1 ; Kai He1 ; Di Wu1 ; Teng Xu1 ; Qixuan Zhang1 ; Kuixiang Shao1 ; Wenzheng Chen 2; Lan Xu1 ; Jingyi Yu1
	2023-06-17
会议录名称	2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
ISSN	1063-6919
卷号	2023-June
页码	4315-4327
发表状态	已发表
DOI	10.1109/CVPR52729.2023.00420
摘要	Human modeling and relighting are two fundamental problems in computer vision and graphics, where high-quality datasets can largely facilitate related research. However, most existing human datasets only provide multi-view human images captured under the same illumination. Although valuable for modeling tasks, they are not read-ily used in relighting problems. To promote research in both fields, in this paper, we present UltraStage, a new 3D human dataset that contains more than 2, 000 high-quality human assets captured under both multi-view and multi-illumination settings. Specifically, for each example, we provide 32 surrounding views illuminated with one white light and two gradient illuminations. In addition to regular multi-view images, gradient illuminations help recover de-tailed surface normal and spatially-varying material maps, enabling various relighting applications. Inspired by recent advances in neural representation, we further interpret each example into a neural human asset which allows novel view synthesis under arbitrary lighting conditions. We show our neural human assets can achieve extremely high capture performance and are capable of representing fine details such as facial wrinkles and cloth folds. We also validate UltraStage in single image relighting tasks, training neural networks with virtual relighted data from neural assets and demonstrating realistic rendering improvements over prior arts. UltraStage will be publicly available to the community to stimulate significant future developments in various human modeling and rendering tasks. The dataset is available at https://miaoing.github.io/RNHA. © 2023 IEEE.
会议录编者/会议主办者	Amazon Science ; Ant Research ; Cruise ; et al. ; Google ; Lambda
关键词	Humans: Face body pose gesture movement
会议名称	2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023
会议地点	Vancouver, BC, Canada
会议日期	17-24 June 2023
URL	查看原文
收录类别	EI
语种	英语
出版者	IEEE Computer Society
EI入藏号	20234114868061
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/333408
专题	信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_本科生信息科学与技术学院_PI研究组_许岚组
共同第一作者	Kai He; Di Wu
通讯作者	Jingyi Yu
作者单位	1.ShanghaiTech University 2.University of Toronto
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Taotao Zhou,Kai He,Di Wu,et al. Relightable Neural Human Assets from Multi-view Gradient Illuminations[C]//Amazon Science, Ant Research, Cruise, et al., Google, Lambda:IEEE Computer Society,2023:4315-4327.