ShanghaiTech University Knowledge Management System
Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis | |
2024 | |
会议录名称 | THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 7 |
ISSN | 2159-5399 |
页码 | 7543-7551 |
摘要 | Obtaining large-scale radiology reports can be difficult for medical images due to various reasons, limiting the effectiveness of contrastive pre-training in the medical image domain and underscoring the need for alternative methods. In this paper, we propose eye-tracking as an alternative to text reports, as it allows for the passive collection of gaze signals without disturbing radiologist's routine diagnosis process. By tracking the gaze of radiologists as they read and diagnose medical images, we can understand their visual attention and clinical reasoning. When a radiologist has similar gazes for two medical images, it may indicate semantic similarity for diagnosis, and these images should be treated as positive pairs when pre-training a computer-assisted diagnosis (CAD) network through contrastive learning. Accordingly, we introduce the Medical contrastive Gaze Image Pre-training (McGIP) as a plug-and-play module for contrastive learning frameworks. McGIP uses radiologist's gaze to guide contrastive pre-training. We evaluate our method using two representative types of medical images and two common types of gaze data. The experimental results demonstrate the practicality of McGIP, indicating its high potential for various clinical scenarios and applications. |
会议名称 | 38th AAAI Conference on Artificial Intelligence (AAAI) / 36th Conference on Innovative Applications of Artificial Intelligence / 14th Symposium on Educational Advances in Artificial Intelligence |
出版地 | 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA |
会议地点 | null,Vancouver,CANADA |
会议日期 | FEB 20-27, 2024 |
URL | 查看原文 |
收录类别 | CPCI-S |
语种 | 英语 |
资助项目 | Key R&D Program of Guangdong Province, China[2021B0101420006] |
WOS研究方向 | Computer Science |
WOS类目 | Computer Science, Artificial Intelligence ; Computer Science, Theory & Methods |
WOS记录号 | WOS:001239937300116 |
出版者 | ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE |
EISSN | 2374-3468 |
引用统计 | 正在获取...
|
文献类型 | 会议论文 |
条目标识符 | https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/381322 |
专题 | 生物医学工程学院 信息科学与技术学院_本科生 生物医学工程学院_PI研究组_沈定刚组 生物医学工程学院_PI研究组_王乾组 |
通讯作者 | Shen, Dinggang |
作者单位 | 1.ShanghaiTech Univ, Sch Biomed Engn, Shanghai, Peoples R China 2.ShanghaiTech Univ, State Key Lab Adv Med Mat & Devices, Shanghai, Peoples R China 3.Shanghai Jiao Tong Univ, Sch Biomed Engn, Shanghai, Peoples R China 4.Shanghai United Imaging Intelligence Co Ltd, Shanghai, Peoples R China 5.Shanghai Clin Res & Trial Ctr, Shanghai, Peoples R China |
第一作者单位 | 生物医学工程学院; 上海科技大学 |
通讯作者单位 | 生物医学工程学院; 上海科技大学 |
第一作者的第一单位 | 生物医学工程学院 |
推荐引用方式 GB/T 7714 | Zhao, Zihao,Wang, Sheng,Wang, Qian,et al. Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosis[C]. 2275 E BAYSHORE RD, STE 160, PALO ALTO, CA 94303 USA:ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE,2024:7543-7551. |
条目包含的文件 | ||||||
文件名称/大小 | 文献类型 | 版本类型 | 开放类型 | 使用许可 |
修改评论
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。