NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions

doi:10.1109/CVPR52688.2022.00606

	NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions
	Jiang, Yuheng1 ; Jiang, Suyi1 ; Sun, Guoxing1 ; Su, Zhuo 2; Guo, Kaiwen 3; Wu, Minye 4; Yu, Jingyi1,5 ; Xu, Lan1,5
	2022
会议录名称	PROCEEDINGS OF THE IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION
ISSN	1063-6919
卷号	2022-June
页码	6145-6155
发表状态	已发表
DOI	10.1109/CVPR52688.2022.00606
摘要	4D modeling of human-object interactions is critical for numerous applications. However, efficient volumetric capture and rendering of complex interaction scenarios, especially from sparse inputs, remain challenging. In this paper, we propose NeuralHOFusion, a neural approach for volumetric human-object capture and rendering using sparse consumer RGBD sensors. It marries traditional non-rigid fusion with recent neural implicit modeling and blending advances, where the captured humans and objects are layer-wise disentangled. For geometry modeling, we propose a neural implicit inference scheme with non-rigid key-volume fusion, as well as a template-aid robust object tracking pipeline. Our scheme enables detailed and complete geometry generation under complex interactions and occlusions. Moreover, we introduce a layer-wise human-object texture rendering scheme, which combines volumetric and image-based rendering in both spatial and temporal domains to obtain photo-realistic results. Extensive experiments demonstrate the effectiveness and efficiency of our approach in synthesizing photo-realistic free-view results under complex human-object interactions. © 2022 IEEE.
会议名称	2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
出版地	10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
会议地点	New Orleans, LA, United states
会议日期	June 19, 2022 - June 24, 2022
URL	查看原文
收录类别	EI ; CPCI-S
语种	英语
资助项目	Shanghai YangFan Program[21YF1429500] ; Shanghai Local college capacity building program[22010502800]
WOS研究方向	Computer Science ; Imaging Science & Photographic Technology
WOS类目	Computer Science, Artificial Intelligence ; Imaging Science & Photographic Technology
WOS记录号	WOS:000867754206041
出版者	IEEE Computer Society
EI入藏号	20224613119578
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/251940
专题	信息科学与技术学院_博士生信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_许岚组
通讯作者	Jiang, Yuheng
作者单位	1.ShanghaiTech Univ, Shanghai, Peoples R China 2.Tencent, Shenzhen, Peoples R China 3.Meta Res Lab, New York, NY USA 4.Katholieke Univ Leuven, Leuven, Belgium 5.Shanghai Engn Res Ctr Intelligent Vis & Imaging, Shanghai, Peoples R China
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Jiang, Yuheng,Jiang, Suyi,Sun, Guoxing,et al. NeuralHOFusion: Neural Volumetric Rendering under Human-object Interactions[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE Computer Society,2022:6145-6155.