消息
×
loading..
iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering
2021-08-12
会议录名称ARXIV
发表状态已发表
DOIarXiv:2108.05577
摘要

Generating "bullet-time'' effects of human free-viewpoint videos is critical for immersive visual effects and VR/AR experience. Recent neural advances still lack the controllable and interactive bullet-time design ability for human free-viewpoint rendering, especially under the real-time, dynamic and general setting for our trajectory-aware task. To fill this gap, in this paper we propose a neural interactive bullet-time generator (iButter) for photo-realistic human free-viewpoint rendering from dense RGB streams, which enables flexible and interactive design for human bullet-time visual effects. Our iButter approach consists of a real-time preview and design stage as well as a trajectory-aware refinement stage. During preview, we propose an interactive bullet-time design approach by extending the NeRF rendering to a real-time and dynamic setting and getting rid of the tedious per-scene training. To this end, our bullet-time design stage utilizes a hybrid training set, light-weight network design and an efficient silhouette-based sampling strategy. During refinement, we introduce an efficient trajectory-aware scheme within 20 minutes, which jointly encodes the spatial, temporal consistency and semantic cues along the designed trajectory, achieving photo-realistic bullet-time viewing experience of human activities. Extensive experiments demonstrate the effectiveness of our approach for convenient interactive bullet-time design and photo-realistic human free-viewpoint video generation.

关键词free-viewpoint video bullet-time novel view synthesis neural rendering neural representation
会议名称29th ACM International Conference on Multimedia (MM)
出版地1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES
会议地点null,null,ELECTR NETWORK
会议日期OCT 20-24, 2021
URL查看原文
收录类别CPCI-S
语种英语
资助项目NSFC programs[
WOS研究方向Computer Science
WOS类目Computer Science, Software Engineering
WOS记录号PPRN:11895369
出版者ASSOC COMPUTING MACHINERY
文献类型会议论文
条目标识符https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/348552
专题信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_虞晶怡组
信息科学与技术学院_硕士生
信息科学与技术学院_PI研究组_许岚组
作者单位
1.Shanghaitech Univ, Shanghai, Peoples R China
2.ShanghaiTech Univ, Shanghai Engn Res Ctr Intelligent Vis & Imaging, Sch Informat Sci & Technol, Shanghai, Peoples R China
第一作者单位上海科技大学
第一作者的第一单位上海科技大学
推荐引用方式
GB/T 7714
Wang, Liao,Wang, Ziyu,Lin, Pei,et al. iButter: Neural Interactive Bullet Time Generator for Human Free-viewpoint Rendering[C]. 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES:ASSOC COMPUTING MACHINERY,2021.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
个性服务
查看访问统计
谷歌学术
谷歌学术中相似的文章
[Wang, Liao]的文章
[Wang, Ziyu]的文章
[Lin, Pei]的文章
百度学术
百度学术中相似的文章
[Wang, Liao]的文章
[Wang, Ziyu]的文章
[Lin, Pei]的文章
必应学术
必应学术中相似的文章
[Wang, Liao]的文章
[Wang, Ziyu]的文章
[Lin, Pei]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。