Multiview Deformation for Dynamic Human Modeling

doi:10.1109/IECON51785.2023.10311856

	Multiview Deformation for Dynamic Human Modeling
	Luo, Xi1,2,3 ; Li, Yuwei1,2,3 ; Yu, Jingyi1
	2023-10-19
会议录名称	IECON 2023- 49TH ANNUAL CONFERENCE OF THE IEEE INDUSTRIAL ELECTRONICS SOCIETY
ISSN	1553-572X
发表状态	已发表
DOI	10.1109/IECON51785.2023.10311856
摘要	We present a novel multi-view dynamic 3D human reconstruction technique based on model-based shape deformation. Our approach specifically targets at handling challenging cases such as textureless appearance, heavy occlusions, and depth order ambiguity that are problematic to stereo-based techniques. We propose to pose match and shape deform a human template model to avoid meshing the point cloud. To robustly match the template pose with image observations, we present a novel Graph Convolutional Networks (GCN) to gradually filter out erroneous views and impose appropriate weights on the optimal subset for recovering the 3D skeleton and warping the template shape. Next, We use the warped human template to guide the cross-view consistent semantic segmentation. We set out to deform the warped 3D model so that the silhouette of the deformed model best matches the target in respective views while maintaining semantic consistency. Comprehensive experiments on publicly available and our newly generated complex motion datasets show our approach significantly outperforms the state-of-the-art on sparse cameras, textureless regions (e.g., under black clothing), complex motions, etc. © 2023 IEEE.
关键词	image processing 3D human modeling pose estimation semantic-driven shape deformation semantic labeling
会议名称	49th Annual Conference of the IEEE Industrial Electronics Society, IECON 2023
会议地点	Singapore, Singapore
会议日期	16-19 Oct. 2023
URL	查看原文
收录类别	EI
语种	英语
出版者	IEEE Computer Society
EI入藏号	20235015212399
EI主题词	Semantics
EISSN	2577-1647
EI分类号	722 Computer Systems and Equipment ; 723 Computer Software, Data Handling and Applications ; 723.2 Data Processing and Image Processing ; 723.4 Artificial Intelligence ; 723.5 Computer Applications ; 741.2 Vision
原始文献类型	Conference article (CA)
来源库	IEEE
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/347928
专题	信息科学与技术学院信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_博士生
作者单位	1.School of Information Science and Technology, ShanghaiTech University, China; 2.University of Chinese Academy of Sciences, China; 3.Shanghai Institute of Microsystem and Information Technology, China
第一作者单位	信息科学与技术学院
第一作者的第一单位	信息科学与技术学院
推荐引用方式 GB/T 7714	Luo, Xi,Li, Yuwei,Yu, Jingyi. Multiview Deformation for Dynamic Human Modeling[C]:IEEE Computer Society,2023.