ShanghaiTech University Knowledge Management System
Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting | |
2022-06-30 | |
会议录名称 | PROCEEDINGS OF THE 36TH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, AAAI 2022 |
卷号 | 36 |
页码 | 3152-3160 |
发表状态 | 已发表 |
摘要 | Recent neural rendering techniques have greatly benefited image-based modeling and relighting tasks. They provide a continuous, compact, and parallelable representation by modeling the plenoptic function as multilayer perceptrons (MLPs). However, vanilla MLPs suffer from spectral biases on multidimensional datasets. Recent rescues based on isotropic Fourier features mapping mitigate the problem but still fall short of handling heterogeneity across different dimensions, causing imbalanced regression and visual artifacts such as excessive blurs. We present an anisotropic random Fourier features (RFF) mapping scheme to tackle spectral biases. We first analyze the influence of bandwidth from a different perspective: we show that the optimal bandwidth exhibits strong correlations with the frequency spectrum of the training data across various dimensions. We then introduce an anisotropic feature mapping scheme with multiple bandwidths to model the multidimensional signal characteristics. We further propose an efficient bandwidth searching scheme through iterative golden-section search that can significantly reduce the training overload from polynomial time to logarithm. Our anisotropic scheme directly applies to neural surface light-field rendering and image-based relighting. Comprehensive experiments show that our scheme can more faithfully model lighting conditions and object features as well as preserve fine texture details and smooth view transitions even when angular and spatial samples are highly imbalanced. Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. |
会议录编者/会议主办者 | Association for the Advancement of Artificial Intelligence |
关键词 | nisotropy Artificial intelligence Bandwidth Fourier transforms Iterative methods Polynomial approximation Rendering (computer graphics) Feature mapping Fourier features Image-based models Image-based relighting Image-Based Rendering Isotropics Mapping scheme Multi-dimensional datasets Multilayers perceptrons Plenoptic functions |
会议名称 | 36th AAAI Conference on Artificial Intelligence, AAAI 2022 |
会议地点 | Virtual, Online |
会议日期 | February 22, 2022 - March 1, 2022 |
收录类别 | EI |
语种 | 英语 |
出版者 | Association for the Advancement of Artificial Intelligence |
EI入藏号 | 20230713571986 |
EI主题词 | Textures |
EI分类号 | 716.1 Information Theory and Signal Processing ; 723.2 Data Processing and Image Processing ; 723.4 Artificial Intelligence ; 723.5 Computer Applications ; 921.3 Mathematical Transformations ; 921.6 Numerical Methods ; 931.2 Physical Properties of Gases, Liquids and Solids |
原始文献类型 | Conference article (CA) |
文献类型 | 会议论文 |
条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/282093 |
专题 | 信息科学与技术学院_硕士生 信息科学与技术学院_PI研究组_虞晶怡组 信息科学与技术学院_PI研究组_邵子瑜组 信息科学与技术学院_博士生 信息科学与技术学院_PI研究组_许岚组 |
作者单位 | 1.School of Information Science and Technology, ShanghaiTech University, China; 2.Shanghai Engineering Research Center of Intelligent Vision and Imaging, China; 3.University of Chinese Academy of Sciences, China; 4.Shanghai Institute of Microsystem and Information Technology, China |
第一作者单位 | 信息科学与技术学院 |
第一作者的第一单位 | 信息科学与技术学院 |
推荐引用方式 GB/T 7714 | Yu, Huangjie,Chen, Anpei,Chen, Xin,et al. Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting[C]//Association for the Advancement of Artificial Intelligence:Association for the Advancement of Artificial Intelligence,2022:3152-3160. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 |
修改评论
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。