See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data

doi:10.1109/ICCV51070.2023.01981

	See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data
	Yuhang Lu1 ; Qi Jiang1 ; Runnan Chen 2; Yuenan Hou 3; Xinge Zhu 4; Yuexin Ma1
	2023
会议录名称	2023 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV)
ISSN	1550-5499
页码	21617-21627
发表状态	已发表
DOI	10.1109/ICCV51070.2023.01981
摘要	Zero-shot point cloud segmentation aims to make deep models capable of recognizing novel objects in point cloud that are unseen in the training phase. Recent trends favor the pipeline which transfers knowledge from seen classes with labels to unseen classes without labels. They typically align visual features with semantic features obtained from word embedding by the supervision of seen classes' annotations. However, point cloud contains limited information to fully match with semantic features. In fact, the rich appearance information of images is a natural complement to the textureless point cloud, which is not well explored in previous literature. Motivated by this, we propose a novel multi-modal zero-shot learning method to better utilize the complementary information of point clouds and images for more accurate visual-semantic alignment. Extensive experiments are performed in two popular benchmarks, i.e., SemanticKITTI and nuScenes, and our method outperforms current SOTA methods with 52% and 49% improvement on average for unseen class mIoU, respectively. © 2023 IEEE.
关键词	Point cloud compression Training Visualization Computer vision Zero-shot learning Semantic segmentation Semantics
会议名称	2023 IEEE/CVF International Conference on Computer Vision, ICCV 2023
会议地点	Paris, France
会议日期	1-6 Oct. 2023
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20241215793211
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	被引频次：14[WOS] [WOS记录] [WOS相关记录]
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/354924
专题	信息科学与技术学院_PI研究组_马月昕信息科学与技术学院_硕士生
共同第一作者	Qi Jiang
通讯作者	Yuexin Ma
作者单位	1.ShanghaiTech University 2.The University of Hong Kong 3.Shanghai AI Laboratory 4.The Chinese University of Hong Kong
第一作者单位	上海科技大学
通讯作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Yuhang Lu,Qi Jiang,Runnan Chen,et al. See More and Know More: Zero-shot Point Cloud Segmentation via Multi-modal Visual Data[C]:Institute of Electrical and Electronics Engineers Inc.,2023:21617-21627.