Holistic Evaluation of GPT-4V for Biomedical Imaging

doi:arXiv:2312.05256

	Holistic Evaluation of GPT-4V for Biomedical Imaging
	Liu, Zhengliang ; Jiang, Hanqi ; Zhong, Tianyang ; Wu, Zihao ; Ma, Chong ; Li, Yiwei ; Yu, Xiaowei ; Zhang, Yutong ; Pan, Yi ; Shu, Peng ; Lyu, Yanjun ; Zhang, Lu ; Yao, Junjie ; Dong, Peixin ; Cao, Chao ; Xiao, Zhenxiang ; Wang, Jiaqi ; Zhao, Huan ; Xu, Shaochen ; Wei, Yaonai ; Chen, Jingyuan ; Dai, Haixing ; Wang, Peilong ; He, Hao ; Wang, Zewei ; Wang, Xinyu ; Zhang, Xu ; Zhao, Lin ; Liu, Yiheng ; Zhang, Kai ; Yan, Liheng ; Sun, Lichao ; Liu, Jun ; Qiang, Ning ; Ge, Bao ; Cai, Xiaoyan ; Zhao, Shijie ; Hu, Xintao ; Yuan, Yixuan ; Li, Gang ; Zhang, Shu ; Zhang, Xin ; Jiang, Xi ; Zhang, Tuo ; Shen, Dinggang ; Li, Quanzheng ; Liu, Wei ; Li, Xiang ; Zhu, Dajiang ; Liu, Tianming
	2023-11-10
状态	已发表
摘要	In this paper, we present a large-scale evaluation probing GPT-4V’s capabilities and limitations for biomedical image analysis. GPT-4V represents a breakthrough in artificial general intelligence (AGI) for computer vision, with applications in the biomedical domain. We assess GPT-4V’s performance across 16 medical imaging categories, including radiology, oncology, ophthalmology, pathology, and more. Tasks include modality recognition, anatomy localization, disease diagnosis, report generation, and lesion detection. The extensive experiments provide insights into GPT4V’s strengths and weaknesses. Results show GPT-4V’s proficiency in modality and anatomy recognition but difficulty with disease diagnosis and localization. GPT-4V excels at diagnostic report generation, indicating strong image captioning skills. While promising for biomedical imaging AI, GPT-4V requires further enhancement and validation before clinical deployment. We emphasize responsible development and testing for trustworthy integration of biomedical AGI. This rigorous evaluation of GPT-4V on diverse medical images advances understanding of multimodal large language models (LLMs) and guides future work toward impactful healthcare applications.
语种	英语
DOI	arXiv:2312.05256
相关网址	查看原文
出处	Arxiv
收录类别	PPRN.PPRN
WOS记录号	PPRN:86539714
WOS类目	Computer Science, Artificial Intelligence ; Engineering, Electrical& Electronic
文献类型	预印本
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/483968
专题	生物医学工程学院
作者单位	1.School of Computing, University of Georgia, GA, USA 2.School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044, China 3.School of Automation, Northwestern Polytechnical University, Xi’an 710072, China 4.Department of Computer Science and Engineering, University of Texas at Arlington, TX, USA 5.Institute of Medical Research, Northwestern Polytechnical University, Xi’an 710072, China China, Chengdu 611731, China 6.School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China 7.Department of Radiation Oncology, Mayo Clinic, Phoenix, Arizona, USA 8.School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710119, China 9.Department of Computer Science and Engineering, Lehigh University, PA, USA 10.Department of Radiology, Second Xiangya Hospital, Central South University, Changsha, 11.Department of Electronic Engineering, Chinese University of Hong Kong, Hong Kong 12.Department of Radiology and BRIC, University of North Carolina at Chapel Hill, NC, USA 13.School of Biomedical Engineering, ShanghaiTech University, and Shanghai Clinical 15 Shanghai United Imaging Intelligence Co., Ltd. 14.Shanghai Clinical Research and Trial Center
推荐引用方式 GB/T 7714	Liu, Zhengliang,Jiang, Hanqi,Zhong, Tianyang,et al. Holistic Evaluation of GPT-4V for Biomedical Imaging. 2023.