消息
×
loading..
I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions
2024-03-30
状态已发表
摘要

We are living in a world surrounded by diverse and "smart" devices with rich modalities of sensing ability. Conveniently capturing the interactions between us humans and these objects remains far-reaching. In this paper, we present I'm-HOI, a monocular scheme to faithfully capture the 3D motions of both the human and object in a novel setting: using a minimal amount of RGB camera and object-mounted Inertial Measurement Unit (IMU). It combines general motion inference and category-aware refinement. For the former, we introduce a holistic human-object tracking method to fuse the IMU signals and the RGB stream and progressively recover the human motions and subsequently the companion object motions. For the latter, we tailor a category-aware motion diffusion model, which is conditioned on both the raw IMU observations and the results from the previous stage under over-parameterization representation. It significantly refines the initial results and generates vivid body, hand, and object motions. Moreover, we contribute a large dataset with ground truth human and object motions, dense RGB inputs, and rich object-mounted IMU measurements. Extensive experiments demonstrate the effectiveness of I'm-HOI under a hybrid capture setting. Our dataset and code will be released to the community.

DOIarXiv:2312.08869
相关网址查看原文
出处Arxiv
WOS记录号PPRN:86592444
WOS类目Computer Science, Software Engineering
资助项目National Key R&D Program of China[
文献类型预印本
条目标识符https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/372936
专题信息科学与技术学院_PI研究组_许岚组
信息科学与技术学院_PI研究组_虞晶怡组
信息科学与技术学院_硕士生
信息科学与技术学院_本科生
信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_汪婧雅组
通讯作者Xu, Lan
作者单位
1.ShanghaiTech Univ, Shanghai, Peoples R China
2.Chinese Acad Sci, Shanghai Adv Res Inst, Shanghai, Peoples R China
3.Univ Chinese Acad Sci, Beijing, Peoples R China
推荐引用方式
GB/T 7714
Zhao, Chengfeng,Zhang, Juze,Du, Jiashen,et al. I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions. 2024.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
个性服务
查看访问统计
谷歌学术
谷歌学术中相似的文章
[Zhao, Chengfeng]的文章
[Zhang, Juze]的文章
[Du, Jiashen]的文章
百度学术
百度学术中相似的文章
[Zhao, Chengfeng]的文章
[Zhang, Juze]的文章
[Du, Jiashen]的文章
必应学术
必应学术中相似的文章
[Zhao, Chengfeng]的文章
[Zhang, Juze]的文章
[Du, Jiashen]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。