ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering

doi:10.1145/3550454.3555505

	ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering
	Rao, Chaolin1,2 ; Yu, Huangjie1 ; Wan, Haochuan1 ; Zhou, Jindong1 ; Zheng, Yueyang1 ; Wu, Minye 3; Ma, Yu1 ; Chen, Anpei1 ; Yuan, Binzhe1 ; Zhou, Pingqiang1 ; Lou, Xin1,2 ; Yu, Jingyi1
	2022-12-01
发表期刊	ACM TRANSACTIONS ON GRAPHICS (IF:7.8[JCR-2023],9.5[5-Year])
ISSN	0730-0301
EISSN	1557-7368
卷号	41 期号:6
发表状态	已发表
DOI	10.1145/3550454.3555505
摘要	The practical deployment of Neural Radiance Fields (NeRF) in rendering applications faces several challenges, with the most critical one being low rendering speed on even high-end graphic processing units (GPUs). In this paper, we present ICARUS, a specialized accelerator architecture tailored for NeRF rendering. Unlike GPUs using general purpose computing and memory architectures for NeRF, ICARUS executes the complete NeRF pipeline using dedicated plenoptic cores (PLCore) consisting of a positional encoding unit (PEU), a multi-layer perceptron (MLP) engine, and a volume rendering unit (VRU). A PLCore takes in positions & directions and renders the corresponding pixel colors without any intermediate data going off-chip for temporary storage and exchange, which can be time and power consuming. To implement the most expensive component of NeRF, i.e., the MLP, we transform the fully connected operations to approximated reconfigurable multiple constant multiplications (MCMs), where common subexpressions are shared across different multiplications to improve the computation efficiency. We build a prototype ICARUS using Synopsys HAPS-80 S104, a field programmable gate array (FPGA)-based prototyping system for large-scale integrated circuits and systems design. We evaluate the power-performance-area (PPA) of a PLCore using 40nm LP CMOS technology. Working at 400 MHz, a single PLCore occupies 16.5 mm(2) and consumes 282.8 mW, translating to 0.105 uJ/sample. The results are compared with those of GPU and tensor processing unit (TPU) implementations.
关键词	Neural radiance fields (NeRF) Neural rendering hardware accelerator
URL	查看原文
收录类别	SCI ; CPCI-S
语种	英语
资助项目	Central Guided Local Science and Technology Foundation of China[YDZX202231000010001]
WOS研究方向	Computer Science
WOS类目	Computer Science, Software Engineering
WOS记录号	WOS:000891651900054
出版者	ASSOC COMPUTING MACHINERY
引用统计	正在获取...
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/266603
专题	信息科学与技术学院_博士生信息科学与技术学院_PI研究组_娄鑫组信息科学与技术学院_PI研究组_虞晶怡组信息科学与技术学院_PI研究组_周平强组信息科学与技术学院_硕士生
通讯作者	Lou, Xin; Yu, Jingyi
作者单位	1.ShanghaiTech Univ, Shanghai, Peoples R China 2.GGU Technol Co Ltd, Shanghai, Peoples R China 3.Katholieke Univ Leuven, Leuven, Belgium
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Rao, Chaolin,Yu, Huangjie,Wan, Haochuan,et al. ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering[J]. ACM TRANSACTIONS ON GRAPHICS,2022,41(6).
APA	Rao, Chaolin.,Yu, Huangjie.,Wan, Haochuan.,Zhou, Jindong.,Zheng, Yueyang.,...&Yu, Jingyi.(2022).ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering.ACM TRANSACTIONS ON GRAPHICS,41(6).
MLA	Rao, Chaolin,et al."ICARUS: A Specialized Architecture for Neural Radiance Fields Rendering".ACM TRANSACTIONS ON GRAPHICS 41.6(2022).