ShanghaiTech University Knowledge Management System
| MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow | |
| 2025-02-17 | |
| 状态 | 已发表 |
| 摘要 | In this paper, we present MVTokenFlow for high-quality 4D content creation from monocular videos. Recent advancements in generative models such as video diffusion models and multiview diffusion models enable us to create videos or 3D models. However, extending these generative models for dynamic 4D content creation is still a challenging task that requires the generated content to be consistent spatially and temporally. To address this challenge, MVTokenFlow utilizes the multiview diffusion model to generate multiview images on different timesteps, which attains spatial consistency across different viewpoints and allows us to reconstruct a reasonable coarse 4D field. Then, MVTokenFlow further regenerates all the multiview images using the rendered 2D flows as guidance. The 2D flows effectively associate pixels from different timesteps and improve the temporal consistency by reusing tokens in the regeneration process. Finally, the regenerated images are spatiotemporally consistent and utilized to refine the coarse 4D field to get a high-quality 4D field. Experiments demonstrate the effectiveness of our design and show significantly improved quality than baseline methods. |
| 语种 | 英语 |
| DOI | arXiv:2502.11697 |
| 相关网址 | 查看原文 |
| 出处 | Arxiv |
| 收录类别 | PPRN.PPRN |
| WOS记录号 | PPRN:121698959 |
| WOS类目 | Computer Science, Software Engineering |
| 资助项目 | National Natural Science Foundation of China[62206174] |
| 文献类型 | 预印本 |
| 条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/514099 |
| 专题 | 信息科学与技术学院_博士生 |
| 通讯作者 | Yang, Sibei |
| 作者单位 | 1.ShanghaiTech Univ, Shanghai, Peoples R China 2.Hong Kong Univ Sci & Technol, Hong Kong, Peoples R China 3.Univ Hong Kong, Hong Kong, Peoples R China |
| 推荐引用方式 GB/T 7714 | Huang, Hanzhuo,Liu, Yuan,Zheng, Ge,et al. MVTokenFlow: High-quality 4D Content Generation using Multiview Token Flow. 2025. |
| 条目包含的文件 | ||||||
| 条目无相关文件。 | ||||||
修改评论
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。