SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos
2021-10
Source PublicationINTERNATIONAL JOURNAL OF COMPUTER VISION
ISSN0920-5691
EISSN1573-1405
Volume129Issue:10Pages:2846-2864
DOI10.1007/s11263-021-01486-4
AbstractMarkerless motion capture and understanding of professional non-daily human movements is an important yet unsolved task, which suffers from complex motion patterns and severe self-occlusion, especially for the monocular setting. In this paper, we propose SportsCap-the first approach for simultaneously capturing 3D human motions and understanding fine-grained actions from monocular challenging sports video input. Our approach utilizes the semantic and temporally structured sub-motion prior in the embedding space for motion capture and understanding in a data-driven multi-task manner. To enable robust capture under complex motion patterns, we propose an effective motion embedding module to recover both the implicit motion embedding and explicit 3D motion details via a corresponding mapping function as well as a sub-motion classifier. Based on such hybrid motion information, we introduce a multi-stream spatial-temporal graph convolutional network to predict the fine-grained semantic action attributes, and adopt a semantic attribute mapping block to assemble various correlated action attributes into a high-level action label for the overall detailed understanding of the whole sequence, so as to enable various applications like action assessment or motion scoring. Comprehensive experiments on both public and our proposed datasets show that with a challenging monocular sports video input, our novel approach not only significantly improves the accuracy of 3D human motion capture, but also recovers accurate fine-grained semantic action attribute.
KeywordHuman modeling 3D motion capture Motion understanding
URL查看原文
Indexed BySCIE ; EI
Language英语
WOS Research AreaComputer Science
WOS SubjectComputer Science, Artificial Intelligence
WOS IDWOS:000681171700003
PublisherSPRINGER
Original Document TypeArticle; Early Access
Citation statistics
Document Type期刊论文
Identifierhttps://kms.shanghaitech.edu.cn/handle/2MSLDSTB/127834
Collection信息科学与技术学院_PI研究组_马月昕
信息科学与技术学院_PI研究组_虞晶怡组
信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_许岚组
Corresponding AuthorXu, Lan; Yu, Jingyi
Affiliation
1.ShanghaiTech Univ, Sch Informat Sci & Technol, Shanghai Engn Res Ctr Intelligent Vis & Imaging, Shanghai, Peoples R China;
2.Chinese Acad Sci, Shanghai Inst Microsyst & Informat Technol, Shanghai, Peoples R China;
3.Univ Chinese Acad Sci, Shanghai, Peoples R China;
4.Huazhong Univ Sci & Technol, Sch Comp Sci & Technol, Wuhan, Peoples R China
First Author AffilicationSchool of Information Science and Technology
Corresponding Author AffilicationSchool of Information Science and Technology
First Signature AffilicationSchool of Information Science and Technology
Recommended Citation
GB/T 7714
Chen, Xin,Pang, Anqi,Yang, Wei,et al. SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2021,129(10):2846-2864.
APA Chen, Xin,Pang, Anqi,Yang, Wei,Ma, Yuexin,Xu, Lan,&Yu, Jingyi.(2021).SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos.INTERNATIONAL JOURNAL OF COMPUTER VISION,129(10),2846-2864.
MLA Chen, Xin,et al."SportsCap: Monocular 3D Human Motion Capture and Fine-Grained Understanding in Challenging Sports Videos".INTERNATIONAL JOURNAL OF COMPUTER VISION 129.10(2021):2846-2864.
Files in This Item: Download All
File Name/Size DocType Version Access License
Related Services
Usage statistics
Scholar Google
Similar articles in Scholar Google
[Chen, Xin]'s Articles
[Pang, Anqi]'s Articles
[Yang, Wei]'s Articles
Baidu academic
Similar articles in Baidu academic
[Chen, Xin]'s Articles
[Pang, Anqi]'s Articles
[Yang, Wei]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[Chen, Xin]'s Articles
[Pang, Anqi]'s Articles
[Yang, Wei]'s Articles
Terms of Use
No data!
Social Bookmark/Share
File name: 10.1007@s11263-021-01486-4.pdf
Format: Adobe PDF
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.