ShanghaiTech University Knowledge Management System
| SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos | |
| 2021-10 | |
| Source Publication | INTERNATIONAL JOURNAL OF COMPUTER VISION
 |
| ISSN | 0920-5691 |
| EISSN | 1573-1405 |
| Volume | 129Issue:10Pages:2846-2864 |
| DOI | 10.1007/s11263-021-01486-4 |
| Abstract | Markerless motion capture and understanding of professional non-daily human movements is an important yet unsolved task, which suffers from complex motion patterns and severe self-occlusion, especially for the monocular setting. In this paper, we propose SportsCap-the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input. Our approach utilizes the semantic and temporally structured sub-motion prior in the embedding space for motion capture and understanding in a data-driven multi-task manner. To enable robust capture under complex motion patterns, we propose an effective motion embedding module to recover both the implicit motion embedding and explicit 3D motion details via a corresponding mapping function as well as a sub-motion classifier. Based on such hybrid motion information, we introduce a multi-stream spatial-temporal graph convolutional network to predict the fine-grained semantic action attributes, and adopt a semantic attribute mapping block to assemble various correlated action attributes into a high-level action label for the overall detailed understanding of the whole sequence, so as to enable various applications like action assessment or motion scoring. Comprehensive experiments on both public and our proposed datasets show that with a challenging monocular sports video input, our novel approach not only significantly improves the accuracy of 3D human motion capture, but also recovers accurate fine-grained semantic action attribute. |
| Keyword | Human modeling 3D motion capture Motion understanding |
| URL | 查看原文 |
| Indexed By | SCIE ; EI |
| Language | 英语 |
| WOS Research Area | Computer Science |
| WOS Subject | Computer Science, Artificial Intelligence |
| WOS ID | WOS:000681171700003 |
| Publisher | SPRINGER |
| Original Document Type | Article; Early Access |
| Citation statistics | |
| Document Type | 期刊论文 |
| Identifier | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/127834 |
| Collection | 信息科学与技术学院_PI研究组_马月昕 信息科学与技术学院_PI研究组_虞晶怡组 信息科学与技术学院_博士生 信息科学与技术学院_PI研究组_许岚组 |
| Corresponding Author | Xu, Lan; Yu, Jingyi |
| Affiliation | 1.ShanghaiTech Univ, Sch Informat Sci & Technol, Shanghai Engn Res Ctr Intelligent Vis & Imaging, Shanghai, Peoples R China; 2.Chinese Acad Sci, Shanghai Inst Microsyst & Informat Technol, Shanghai, Peoples R China; 3.Univ Chinese Acad Sci, Shanghai, Peoples R China; 4.Huazhong Univ Sci & Technol, Sch Comp Sci & Technol, Wuhan, Peoples R China |
| First Author Affilication | School of Information Science and Technology |
| Corresponding Author Affilication | School of Information Science and Technology |
| First Signature Affilication | School of Information Science and Technology |
| Recommended Citation GB/T 7714 | Chen, Xin,Pang, Anqi,Yang, Wei,et al. SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2021,129(10):2846-2864. |
| APA | Chen, Xin,Pang, Anqi,Yang, Wei,Ma, Yuexin,Xu, Lan,&Yu, Jingyi.(2021).SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos.INTERNATIONAL JOURNAL OF COMPUTER VISION,129(10),2846-2864. |
| MLA | Chen, Xin,et al."SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos".INTERNATIONAL JOURNAL OF COMPUTER VISION 129.10(2021):2846-2864. |
| Files in This Item: | Download All | |||||
| File Name/Size | DocType | Version | Access | License | ||
Edit Comment
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.