Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting
2022-06-30
会议录名称PROCEEDINGS OF THE 36TH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, AAAI 2022
卷号36
页码3152-3160
发表状态已发表
摘要

Recent neural rendering techniques have greatly benefited image-based modeling and relighting tasks. They provide a continuous, compact, and parallelable representation by modeling the plenoptic function as multilayer perceptrons (MLPs). However, vanilla MLPs suffer from spectral biases on multidimensional datasets. Recent rescues based on isotropic Fourier features mapping mitigate the problem but still fall short of handling heterogeneity across different dimensions, causing imbalanced regression and visual artifacts such as excessive blurs. We present an anisotropic random Fourier features (RFF) mapping scheme to tackle spectral biases. We first analyze the influence of bandwidth from a different perspective: we show that the optimal bandwidth exhibits strong correlations with the frequency spectrum of the training data across various dimensions. We then introduce an anisotropic feature mapping scheme with multiple bandwidths to model the multidimensional signal characteristics. We further propose an efficient bandwidth searching scheme through iterative golden-section search that can significantly reduce the training overload from polynomial time to logarithm. Our anisotropic scheme directly applies to neural surface light-field rendering and image-based relighting. Comprehensive experiments show that our scheme can more faithfully model lighting conditions and object features as well as preserve fine texture details and smooth view transitions even when angular and spatial samples are highly imbalanced. Copyright © 2022, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

会议录编者/会议主办者Association for the Advancement of Artificial Intelligence
关键词nisotropy Artificial intelligence Bandwidth Fourier transforms Iterative methods Polynomial approximation Rendering (computer graphics) Feature mapping Fourier features Image-based models Image-based relighting Image-Based Rendering Isotropics Mapping scheme Multi-dimensional datasets Multilayers perceptrons Plenoptic functions
会议名称36th AAAI Conference on Artificial Intelligence, AAAI 2022
会议地点Virtual, Online
会议日期February 22, 2022 - March 1, 2022
收录类别EI
语种英语
出版者Association for the Advancement of Artificial Intelligence
EI入藏号20230713571986
EI主题词Textures
EI分类号716.1 Information Theory and Signal Processing ; 723.2 Data Processing and Image Processing ; 723.4 Artificial Intelligence ; 723.5 Computer Applications ; 921.3 Mathematical Transformations ; 921.6 Numerical Methods ; 931.2 Physical Properties of Gases, Liquids and Solids
原始文献类型Conference article (CA)
文献类型会议论文
条目标识符https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/282093
专题信息科学与技术学院_硕士生
信息科学与技术学院_PI研究组_虞晶怡组
信息科学与技术学院_PI研究组_邵子瑜组
信息科学与技术学院_博士生
信息科学与技术学院_PI研究组_许岚组
作者单位
1.School of Information Science and Technology, ShanghaiTech University, China;
2.Shanghai Engineering Research Center of Intelligent Vision and Imaging, China;
3.University of Chinese Academy of Sciences, China;
4.Shanghai Institute of Microsystem and Information Technology, China
第一作者单位信息科学与技术学院
第一作者的第一单位信息科学与技术学院
推荐引用方式
GB/T 7714
Yu, Huangjie,Chen, Anpei,Chen, Xin,et al. Anisotropic Fourier Features for Neural Image-Based Rendering and Relighting[C]//Association for the Advancement of Artificial Intelligence:Association for the Advancement of Artificial Intelligence,2022:3152-3160.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可
个性服务
查看访问统计
谷歌学术
谷歌学术中相似的文章
[Yu, Huangjie]的文章
[Chen, Anpei]的文章
[Chen, Xin]的文章
百度学术
百度学术中相似的文章
[Yu, Huangjie]的文章
[Chen, Anpei]的文章
[Chen, Xin]的文章
必应学术
必应学术中相似的文章
[Yu, Huangjie]的文章
[Chen, Anpei]的文章
[Chen, Xin]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
暂无评论
 

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。