An Efficient Hardware Volume Renderer for Convolutional Neural Radiance Fields

doi:10.1109/ISCAS58744.2024.10558395

	An Efficient Hardware Volume Renderer for Convolutional Neural Radiance Fields
	Wang XX(王学鑫)1 ; He YX(贺云翔)1 ; Zhang XY(张湘煜)1 ; Zhou PQ(周平强)1 ; Lou X(娄鑫)1,2
	2024-05
会议录名称	2024 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS
ISSN	0271-4302
发表状态	已发表
DOI	10.1109/ISCAS58744.2024.10558395
摘要	Neural Radiance Fields (NeRF) has attracted growing attention in the fields of 3D reconstruction and rendering. However, straightforward NeRF algorithms encounter challenges in accurately capturing complex surface details with rich high-frequency information. A recent development known as Convolutional Neural Radiance Field (ConvNeRF) has demonstrated state-of-the-art results for these tasks. But it comes with substantial irregular computational requirements, particularly in the convolutional volume rendering phase. In this paper, we introduce a hardware accelerator designed to enhance the efficiency of convolutional volume rendering in ConvNeRF. Our approach includes the creation of specialized computation modules and corresponding on-chip memory system optimized for seamless support of gated convolutions and skip connections in ConvNeRF. To validate our design, we implement it in VerilogHDL and build a prototype using Field Programmable Gate Array (FPGA). We also map our design to 40nm CMOS technology. The evaluation results underscore the superiority of our accelerator in terms of energy efficiency when compared to an implementation on an NVIDIA 2080Ti GPU, offering approximately 84.6× more frames per watt.
会议录编者/会议主办者	Agency for Science, Technology and Research, Institute of Microelectronics (IME) ; Cadence ; Continental ; et al. ; National University of Singapore, Department of Electrical and Computer Engineering, College of Design and Engineering ; Synopsys
关键词	Energy efficiency Field programmable gate arrays (FPGA) Integrated circuit design Volume rendering 3-D rendering 3D reconstruction Complex surface Computational requirements Convolutional volume rendering Gated convolution High-frequency informations Neural radiance field State of the art Surface details
会议名称	2024 IEEE International Symposium on Circuits and Systems, ISCAS 2024
会议地点	Singapore, Singapore
会议日期	19-22 May 2024
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20242916713984
EI主题词	Convolution
EI分类号	525.2 Energy Conservation ; 714.2 Semiconductor Devices and Integrated Circuits ; 716.1 Information Theory and Signal Processing ; 721.2 Logic Elements ; 723.2 Data Processing and Image Processing ; 723.5 Computer Applications
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/362293
专题	信息科学与技术学院_硕士生信息科学与技术学院_PI研究组_娄鑫组信息科学与技术学院_PI研究组_周平强组信息科学与技术学院_本科生
通讯作者	Lou X(娄鑫)
作者单位	1.上海科技大学信息科学与技术学院 2.智能感知与人机协同教育部重点实验室
第一作者单位	信息科学与技术学院
通讯作者单位	信息科学与技术学院
第一作者的第一单位	信息科学与技术学院
推荐引用方式 GB/T 7714	Wang XX,He YX,Zhang XY,et al. An Efficient Hardware Volume Renderer for Convolutional Neural Radiance Fields[C]//Agency for Science, Technology and Research, Institute of Microelectronics (IME), Cadence, Continental, et al., National University of Singapore, Department of Electrical and Computer Engineering, College of Design and Engineering, Synopsys:Institute of Electrical and Electronics Engineers Inc.,2024.