| |||||||
ShanghaiTech University Knowledge Management System
iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering | |
2021-08-12 | |
会议录名称 | ARXIV |
发表状态 | 已发表 |
DOI | arXiv:2108.05577 |
摘要 | Generating "bullet-time'' effects of human free-viewpoint videos is critical for immersive visual effects and VR/AR experience. Recent neural advances still lack the controllable and interactive bullet-time design ability for human free-viewpoint rendering, especially under the real-time, dynamic and general setting for our trajectory-aware task. To fill this gap, in this paper we propose a neural interactive bullet-time generator (iButter) for photo-realistic human free-viewpoint rendering from dense RGB streams, which enables flexible and interactive design for human bullet-time visual effects. Our iButter approach consists of a real-time preview and design stage as well as a trajectory-aware refinement stage. During preview, we propose an interactive bullet-time design approach by extending the NeRF rendering to a real-time and dynamic setting and getting rid of the tedious per-scene training. To this end, our bullet-time design stage utilizes a hybrid training set, light-weight network design and an efficient silhouette-based sampling strategy. During refinement, we introduce an efficient trajectory-aware scheme within 20 minutes, which jointly encodes the spatial, temporal consistency and semantic cues along the designed trajectory, achieving photo-realistic bullet-time viewing experience of human activities. Extensive experiments demonstrate the effectiveness of our approach for convenient interactive bullet-time design and photo-realistic human free-viewpoint video generation. |
关键词 | free-viewpoint video bullet-time novel view synthesis neural rendering neural representation |
会议名称 | 29th ACM International Conference on Multimedia (MM) |
出版地 | 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES |
会议地点 | null,null,ELECTR NETWORK |
会议日期 | OCT 20-24, 2021 |
URL | 查看原文 |
收录类别 | CPCI-S |
语种 | 英语 |
资助项目 | NSFC programs[ |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Software Engineering |
WOS记录号 | PPRN:11895369 |
出版者 | ASSOC COMPUTING MACHINERY |
文献类型 | 会议论文 |
条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348552 |
专题 | 信息科学与技术学院_博士生 信息科学与技术学院_PI研究组_虞晶怡组 信息科学与技术学院_硕士生 信息科学与技术学院_PI研究组_许岚组 |
作者单位 | 1.Shanghaitech Univ, Shanghai, Peoples R China 2.ShanghaiTech Univ, Shanghai Engn Res Ctr Intelligent Vis & Imaging, Sch Informat Sci & Technol, Shanghai, Peoples R China |
第一作者单位 | 上海科技大学 |
第一作者的第一单位 | 上海科技大学 |
推荐引用方式 GB/T 7714 | Wang, Liao,Wang, Ziyu,Lin, Pei,et al. iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering[C]. 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES:ASSOC COMPUTING MACHINERY,2021. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 |
个性服务 |
查看访问统计 |
谷歌学术 |
谷歌学术中相似的文章 |
[Wang, Liao]的文章 |
[Wang, Ziyu]的文章 |
[Lin, Pei]的文章 |
百度学术 |
百度学术中相似的文章 |
[Wang, Liao]的文章 |
[Wang, Ziyu]的文章 |
[Lin, Pei]的文章 |
必应学术 |
必应学术中相似的文章 |
[Wang, Liao]的文章 |
[Wang, Ziyu]的文章 |
[Lin, Pei]的文章 |
相关权益政策 |
暂无数据 |
收藏/分享 |
修改评论
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。