TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understanding

doi:10.1109/WACV57701.2024.00097

	TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understanding
	Shuo Wang1 ; Jing Li 2; Zibo Zhao1 ; Dongze Lian 3; Binbin Huang1 ; Xiaomei Wang 4; Zhengxin Li1 ; Shenghua Gao1
	2024
会议录名称	2024 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV)
ISSN	2472-6737
页码	914-923
发表状态	已发表
DOI	10.1109/WACV57701.2024.00097
摘要	Holistic scene understanding includes semantic segmentation, surface normal estimation, object boundary detection, depth estimation, etc. The key aspect of this problem is to learn representation effectively, as each subtask builds upon not only correlated but also distinct attributes. Inspired by visual-prompt tuning, we propose a Task-Specific Prompts Transformer, dubbed TSP-Transformer, for holistic scene understanding. It features a vanilla transformer in the early stage and tasks-specific prompts transformer encoder in the lateral stage, where tasks-specific prompts are augmented. By doing so, the transformer layer learns the generic information from the shared parts and is endowed with task-specific capacity. First, the tasks-specific prompts serve as induced priors for each task effectively. Moreover, the task-specific prompts can be seen as switches to favor task-specific representation learning for different tasks. Extensive experiments on NYUD-v2 and PASCAL-Context show that our method achieves state-of-the-art performance, validating the effectiveness of our method for holistic scene understanding. We also provide our code in the following link https://github.com/tb2-sy/TSP-Transformer. © 2024 IEEE.
会议录编者/会议主办者	CVF ; IEEE Computer Society
关键词	Algorithms Image recognition and understanding
会议名称	2024 IEEE Winter Conference on Applications of Computer Vision, WACV 2024
会议地点	Waikoloa, HI, USA
会议日期	3-8 Jan. 2024
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20241816028495
EI主题词	Image recognition
EI分类号	723.2 Data Processing and Image Processing ; 723.4 Artificial Intelligence ; 723.5 Computer Applications ; 741.2 Vision
原始文献类型	Conference article (CA)
来源库	IEEE
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/362292
专题	信息科学与技术学院_PI研究组_高盛华组信息科学与技术学院_硕士生信息科学与技术学院_博士生
作者单位	1.ShanghaiTech University 2.Xiaohongshu Inc. 3.National University of Singapore 4.Fudan University
第一作者单位	上海科技大学
第一作者的第一单位	上海科技大学
推荐引用方式 GB/T 7714	Shuo Wang,Jing Li,Zibo Zhao,et al. TSP-Transformer: Task-Specific Prompts Boosted Transformer for Holistic Scene Understanding[C]//CVF, IEEE Computer Society:Institute of Electrical and Electronics Engineers Inc.,2024:914-923.