Dynamic Grained Encoder for Vision Transformers

doi:暂无

	Dynamic Grained Encoder for Vision Transformers
	Song, Lin 1; Zhang, Songyang2,4,5 ; Liu, Songtao 3; Li, Zeming 3; He, Xuming2 ; Sun, Hongbin 1; Sun, Jian 3; Zheng, Nanning 1
	2021
会议录名称	ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS
ISSN	1049-5258
卷号	7
页码	5770-5783
发表状态	已发表
DOI	暂无
摘要	Transformers, the de-facto standard for language modeling, have been recently applied for vision tasks. This paper introduces sparse queries for vision transformers to exploit the intrinsic spatial redundancy of natural images and save computational costs. Specifically, we propose a Dynamic Grained Encoder for vision transformers, which can adaptively assign a suitable number of queries to each spatial region. Thus it achieves a fine-grained representation in discriminative regions while keeping high efficiency. Besides, the dynamic grained encoder is compatible with most vision transformer frameworks. Without bells and whistles, our encoder allows the state-of-the-art vision transformers to reduce computational complexity by 40%-60% while maintaining comparable performance on image classification. Extensive experiments on object detection and segmentation further demonstrate the generalizability of our approach. Code is available at https://github.com/StevenGrove/vtpack. © 2021 Neural information processing systems foundation. All rights reserved.
关键词	Modeling languages Object detection Computational costs De facto standard Fine grained Higher efficiency Language model Natural images Performance Spatial redundancy Spatial regions State of the art
会议名称	35th Conference on Neural Information Processing Systems, NeurIPS 2021
会议地点	Virtual, Online
会议日期	December 6, 2021 - December 14, 2021
收录类别	EI
语种	英语
出版者	Neural information processing systems foundation
EI入藏号	20222412223132
EI主题词	Signal encoding
EI分类号	716.1 Information Theory and Signal Processing ; 723.2 Data Processing and Image Processing
原始文献类型	Conference article (CA)
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/251938
专题	信息科学与技术学院_PI研究组_何旭明组信息科学与技术学院_博士生
作者单位	1.College of Artificial Intelligence, Xi'an Jiaotong University, China; 2.ShanghaiTech University, China; 3.Megvii Inc. (Face++); 4.University of Chinese Academy of Sciences, China; 5.Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, China
推荐引用方式 GB/T 7714	Song, Lin,Zhang, Songyang,Liu, Songtao,et al. Dynamic Grained Encoder for Vision Transformers[C]:Neural information processing systems foundation,2021:5770-5783.