Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis

	Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis
	Zhao, Zihao1,2 ; Wang, Sheng 1,2,3,4; Wang, Qian1,2,5 ; Shen, Dinggang1,2,4,5
	2024
会议录名称	THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 7
ISSN	2159-5399
页码	7543-7551
摘要	Obtaining large-scale radiology reports can be difficult for medical images due to various reasons, limiting the effectiveness of contrastive pre-training in the medical image domain and underscoring the need for alternative methods. In this paper, we propose eye-tracking as an alternative to text reports, as it allows for the passive collection of gaze signals without disturbing radiologist's routine diagnosis process. By tracking the gaze of radiologists as they read and diagnose medical images, we can understand their visual attention and clinical reasoning. When a radiologist has similar gazes for two medical images, it may indicate semantic similarity for diagnosis, and these images should be treated as positive pairs when pre-training a computer-assisted diagnosis (CAD) network through contrastive learning. Accordingly, we introduce the Medical contrastive Gaze Image Pre-training (McGIP) as a plug-and-play module for contrastive learning frameworks. McGIP uses radiologist's gaze to guide contrastive pre-training. We evaluate our method using two representative types of medical images and two common types of gaze data. The experimental results demonstrate the practicality of McGIP, indicating its high potential for various clinical scenarios and applications.
会议名称	38th AAAI Conference on Artificial Intelligence (AAAI) / 36th Conference on Innovative Applications of Artificial Intelligence / 14th Symposium on Educational Advances in Artificial Intelligence
出版地	2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA
会议地点	null,Vancouver,CANADA
会议日期	FEB 20-27, 2024
URL	查看原文
收录类别	CPCI-S
语种	英语
资助项目	Key R&D Program of Guangdong Province, China[2021B0101420006]
WOS研究方向	Computer Science
WOS类目	Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods
WOS记录号	WOS:001239937300116
出版者	ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE
EISSN	2374-3468
引用统计	正在获取...
文献类型	会议论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/381322
专题	生物医学工程学院信息科学与技术学院_本科生生物医学工程学院_PI研究组_沈定刚组生物医学工程学院_PI研究组_王乾组
通讯作者	Shen, Dinggang
作者单位	1.ShanghaiTech Univ, Sch Biomed Engn, Shanghai, Peoples R China 2.ShanghaiTech Univ, State Key Lab Adv Med Mat & Devices, Shanghai, Peoples R China 3.Shanghai Jiao Tong Univ, Sch Biomed Engn, Shanghai, Peoples R China 4.Shanghai United Imaging Intelligence Co Ltd, Shanghai, Peoples R China 5.Shanghai Clin Res & Trial Ctr, Shanghai, Peoples R China
第一作者单位	生物医学工程学院; 上海科技大学
通讯作者单位	生物医学工程学院; 上海科技大学
第一作者的第一单位	生物医学工程学院
推荐引用方式 GB/T 7714	Zhao, Zihao,Wang, Sheng,Wang, Qian,et al. Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis[C]. 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA:ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE,2024:7543-7551.