UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase

doi:10.1109/ICCV51070.2023.01980

	UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase
	Liu, Youquan 1,2; Chen, Runnan 1,3; Li, Xin 1,4; Kong, Lingdong 1,5; Yang, Yuchen 1,6; Xia, Zhaoyang 1,6; Bai, Yeqi 1; Zhu, Xinge 7; Ma, Yuexin8 ; Li, Yikang 1; Qiao, Yu 1; Hou, Yuenan 1
	2023
会议录名称	PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION
ISSN	1550-5499
页码	21605-21616
发表状态	已发表
DOI	10.1109/ICCV51070.2023.01980
摘要	Point-, voxel-, and range-views are three representative forms of point clouds. All of them have accurate 3D measurements but lack color and texture information. RGB images are a natural complement to these point cloud views and fully utilizing the comprehensive information of them benefits more robust perceptions. In this paper, we present a unified multi-modal LiDAR segmentation network, termed UniSeg, which leverages the information of RGB images and three views of the point cloud, and accomplishes semantic segmentation and panoptic segmentation simultaneously. Specifically, we first design the Learnable cross-Modal Association (LMA) module to automatically fuse voxel-view and range-view features with image features, which fully utilize the rich semantic information of images and are robust to calibration errors. Then, the enhanced voxel-view and range-view features are transformed to the point space, where three views of point cloud features are further fused adaptively by the Learnable cross-View Association module (LVA). Notably, UniSeg achieves promising results in three public benchmarks, i.e., SemanticKITTI, nuScenes, and Waymo Open Dataset (WOD); it ranks 1st on two challenges of two benchmarks, including the LiDAR semantic segmentation challenge of nuScenes and panoptic segmentation challenges of SemanticKITTI. Besides, we construct the OpenPCSeg codebase, which is the largest and most comprehensive outdoor LiDAR segmentation codebase. It contains most of the popular outdoor LiDAR segmentation algorithms and provides reproducible implementations. The OpenPCSeg codebase will be made publicly available at https://github.com/PJLab-ADG/PCSeg. © 2023 IEEE.
会议名称	2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023
出版地	10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA
会议地点	Paris, France
会议日期	October 2, 2023 - October 6, 2023
URL	查看原文
收录类别	EI ; CPCI-S
语种	英语
资助项目	Science and Technology Commission of Shanghai Municipality[22DZ1100102]
WOS研究方向	Computer Science ; Imaging Science & Photographic Technology
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods ; Imaging Science & Photographic Technology
WOS记录号	WOS:001169500506022
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20240915636214
原始文献类型	Conference article (CA)
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/370163
专题	信息科学与技术学院_PI研究组_马月昕
通讯作者	Bai, Yeqi; Li, Yikang; Hou, Yuenan
作者单位	1.Shanghai Ai Laboratory, China 2.Hochschule Bremerhaven, Germany 3.The University of Hong Kong, Hong Kong 4.East China Normal University, China 5.National University of Singapore, Singapore 6.Fudan University, China 7.The Chinese University of Hong Kong, Hong Kong 8.Shanghai Tech University, China
推荐引用方式 GB/T 7714	Liu, Youquan,Chen, Runnan,Li, Xin,et al. UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:Institute of Electrical and Electronics Engineers Inc.,2023:21605-21616.