OSLNet: Deep Small-Sample Classification With an Orthogonal Softmax Layer

doi:10.1109/TIP.2020.2990277

	OSLNet: Deep Small-Sample Classification With an Orthogonal Softmax Layer
	Xiaoxu Li 1; Dongliang Chang 2; Zhanyu Ma 2; Zheng-Hua Tan 3; Jing-Hao Xue 4; Jie Cao 1; Jingyi Yu5 ; Jun Guo 1
	2020
发表期刊	IEEE TRANSACTIONS ON IMAGE PROCESSING (IF:10.8[JCR-2023],12.1[5-Year])
ISSN	1941-0042
卷号	29 页码:6482-6495
发表状态	已发表
DOI	10.1109/TIP.2020.2990277
摘要	A deep neural network of multiple nonlinear layers forms a large function space, which can easily lead to overfitting when it encounters small-sample data. To mitigate overfitting in small-sample classification, learning more discriminative features from small-sample data is becoming a new trend. To this end, this paper aims to find a subspace of neural networks that can facilitate a large decision margin. Specifically, we propose the Orthogonal Softmax Layer (OSL), which makes the weight vectors in the classification layer remain orthogonal during both the training and test processes. The Rademacher complexity of a network using the OSL is only $\frac {1}{K}$ , where $K$ is the number of classes, of that of a network using the fully connected classification layer, leading to a tighter generalization error bound. Experimental results demonstrate that the proposed OSL has better performance than the methods used for comparison on four small-sample benchmark datasets, as well as its applicability to large-sample datasets. Codes are available at: https://github.com/dongliangchang/OSLNet.
关键词	Training Optimization Training data Deep learning Decorrelation Biological neural networks
URL	查看原文
收录类别	EI ; SCIE ; SCI
原始文献类型	Journals
来源库	IEEE
引用统计	正在获取...
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/122198
专题	信息科学与技术学院_PI研究组_虞晶怡组
作者单位	1.School of Computer and Communication, Lanzhou University of Technology, Lanzhou, China 2.Pattern Recognition and Intelligent System Laboratory, School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China 3.Department of Electronic Systems, Aalborg University, Aalborg, Denmark 4.Department of Statistical Science, University College London, London, U.K. 5.School of Information Science and Technology, ShanghaiTech University, Shanghai, China
推荐引用方式 GB/T 7714	Xiaoxu Li,Dongliang Chang,Zhanyu Ma,et al. OSLNet: Deep Small-Sample Classification With an Orthogonal Softmax Layer[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2020,29:6482-6495.
APA	Xiaoxu Li.,Dongliang Chang.,Zhanyu Ma.,Zheng-Hua Tan.,Jing-Hao Xue.,...&Jun Guo.(2020).OSLNet: Deep Small-Sample Classification With an Orthogonal Softmax Layer.IEEE TRANSACTIONS ON IMAGE PROCESSING,29,6482-6495.
MLA	Xiaoxu Li,et al."OSLNet: Deep Small-Sample Classification With an Orthogonal Softmax Layer".IEEE TRANSACTIONS ON IMAGE PROCESSING 29(2020):6482-6495.