Memorizing Structure-Texture Correspondence for Image Anomaly Detection

doi:10.1109/TNNLS.2021.3101403

	Memorizing Structure-Texture Correspondence for Image Anomaly Detection
	Kang Zhou1,2,3 ; Jing Li1,2,3 ; Yuting Xiao1 ; Jianlong Yang 4; Jun Cheng 5; Wen Liu1 ; Weixin Luo1 ; Jiang Liu1,6,7 ; Shenghua Gao1,8,9
	2022-06-01
发表期刊	IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (IF:10.2[JCR-2023],10.4[5-Year])
ISSN	2162-2388
卷号	33 期号:6
发表状态	已发表
DOI	10.1109/TNNLS.2021.3101403
摘要	This work focuses on image anomaly detection by leveraging only normal images in the training phase. Most previous methods tackle anomaly detection by reconstructing the input images with an autoencoder (AE)-based model, and an underlying assumption is that the reconstruction errors for the normal images are small, and those for the abnormal images are large. However, these AE-based methods, sometimes, even reconstruct the anomalies well; consequently, they are less sensitive to anomalies. To conquer this issue, we propose to reconstruct the image by leveraging the structure-texture correspondence. Specifically, we observe that, usually, for normal images, the texture can be inferred from its corresponding structure (e.g., the blood vessels in the fundus image and the structured anatomy in optical coherence tomography image), while it is hard to infer the texture from a destroyed structure for the abnormal images. Therefore, a structure-texture correspondence memory (STCM) module is proposed to reconstruct image texture from its structure, where a memory mechanism is used to characterize the mapping from the normal structure to its corresponding normal texture. As the correspondence between destroyed structure and texture cannot be characterized by the memory, the abnormal images would have a larger reconstruction error, facilitating anomaly detection. In this work, we utilize two kinds of complementary structures (i.e., the semantic structure with human-labeled category information and the low-level structure with abundant details), which are extracted by two structure extractors. The reconstructions from the two kinds of structures are fused together by a learned attention weight to get the final reconstructed image. We further feed the reconstructed image into the two aforementioned structure extractors to extract structures. On the one hand, constraining the consistency between the structures extracted from the original input and that from the reconstructed image would regularize the network training; on the other hand, the error between the structures extracted from the original input and that from the reconstructed image can also be used as a supplement measurement to identify the anomaly. Extensive experiments validate the effectiveness of our method for image anomaly detection on both industrial inspection images and medical images.
URL	查看原文
收录类别	SCI ; EI ; SCIE
来源库	IEEE
引用统计	被引频次：40[WOS] [WOS记录] [WOS相关记录]
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/135788
专题	生物医学工程学院信息科学与技术学院信息科学与技术学院_PI研究组_高盛华组信息科学与技术学院_特聘教授组_刘江组信息科学与技术学院_博士生
作者单位	1.School of Information Science and Technology, ShanghaiTech University, Shanghai, China 2.Chinese Academy of Sciences, Shanghai Institute of Microsystem and Information Technology, Shanghai, China 3.University of Chinese Academy of Sciences, Beijing, China 4.School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China 5.Agency for Science, Technology and Research, Institute for Infocomm Research, Singapore 6.Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China 7.Chinese Academy of Sciences, Cixi Institute of Biomedical Engineering, Ningbo, China 8.Shanghai Engineering Research Center of Intelligent Vision and Imaging, Shanghai, China 9.Shanghai Engineering Research Center of Energy Efficient and Custom AI IC, Shanghai, China
第一作者单位	信息科学与技术学院
第一作者的第一单位	信息科学与技术学院
推荐引用方式 GB/T 7714	Kang Zhou,Jing Li,Yuting Xiao,et al. Memorizing Structure-Texture Correspondence for Image Anomaly Detection[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2022,33(6).
APA	Kang Zhou.,Jing Li.,Yuting Xiao.,Jianlong Yang.,Jun Cheng.,...&Shenghua Gao.(2022).Memorizing Structure-Texture Correspondence for Image Anomaly Detection.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,33(6).
MLA	Kang Zhou,et al."Memorizing Structure-Texture Correspondence for Image Anomaly Detection".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 33.6(2022).