Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task

doi:10.1109/TBDATA.2025.3536928

	Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task
	Wu, Zihao 1; Zhang, Lu 2; Cao, Chao 2; Yu, Xiaowei 2; Liu, Zhengliang 1; Zhao, Lin 1; Li, Yiwei 1; Dai, Haixing 1; Ma, Chong 3; Li, Gang 4; Liu, Wei 5; Li, Quanzheng 6; Shen, Dinggang7,8,9 ; Li, Xiang 6; Zhu, Dajiang 2; Liu, Tianming 1
	2025
发表期刊	IEEE TRANSACTIONS ON BIG DATA (IF:7.5[JCR-2023],5.8[5-Year])
ISSN	2372-2096
EISSN	2332-7790
卷号	PP 期号:99
发表状态	已发表
DOI	10.1109/TBDATA.2025.3536928
摘要	Recently, ChatGPT and GPT-4 have emerged and gained immense global attention due to their unparalleled performance in language processing. Despite demonstrating impressive capability in various open-domain tasks, their adequacy in highly specific fields like radiology remains untested. Radiology presents unique linguistic phenomena distinct from open-domain data due to its specificity and complexity. Assessing the performance of large language models (LLMs) in such specific domains is crucial not only for a thorough evaluation of their overall performance but also for providing valuable insights into future model design directions: whether model design should be generic or domain-specific. To this end, in this study, we evaluate the performance of ChatGPT/GPT-4 on a radiology natural language inference (NLI) task and compare it to other models fine-tuned specifically on task-related data samples. We also conduct a comprehensive investigation on ChatGPT/GPT-4’s reasoning ability by introducing varying levels of inference difficulty. Our results show that 1) ChatGPT and GPT-4 outperform other LLMs in the radiology NLI task; 2) other specifically fine-tuned Bert-based models require significant amounts of data samples to achieve comparable performance to ChatGPT/GPT-4. These findings not only demonstrate the feasibility and promise of constructing a generic model capable of addressing various tasks across different domains, but also highlight several key factors crucial for developing a unified model, particularly in a medical context, paving the way for future artificial general intelligence (AGI) systems. We release our code and data to the research community‡. ©2015 IEEE.
关键词	Inference engines Intelligent systems Network security Problem oriented languages Unified Modeling Language Language inference Language model Language processing Large language model Natural language inference Natural language processing Natural languages Performance Radiology reports
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20250817904048
EI主题词	Natural language processing systems
EI分类号	1101 Artificial Intelligence ; 1101.1 Expert Systems ; 1106 Computer Software, Data Handling and Applications ; 1106.1.1 Computer Programming Languages ; 1106.2 Data Handling and Data Processing ; 1106.7 Computational Linguistics
原始文献类型	Article in Press
来源库	IEEE
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/490328
专题	生物医学工程学院生物医学工程学院_PI研究组_沈定刚组
通讯作者	Zhu, Dajiang; Liu, Tianming
作者单位	1.The School of Computing, University of Georgia, Athens; 30602, United States; 2.the Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington; 76019, United States; 3.The School of Automation, Northwestern Polytechnical University, 710072, China; 4.The Department of Radiology and BRIC, University of North Carolina at Chapel Hill, Chapel Hill; 27599, United States; 5.The Department of Radiation Oncology, Mayo Clinic, Phoenix; 85054, United States; 6.The Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston; 02115, United States; 7.The School of Biomedical Engineering, ShanghaiTech University, Shanghai; 201210, China; 8.Shanghai United Imaging Intelligence Co., Ltd., Shanghai; 200230, China; 9.Shanghai Clinical Research and Trial Center, Shanghai; 201210, China
推荐引用方式 GB/T 7714	Wu, Zihao,Zhang, Lu,Cao, Chao,et al. Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task[J]. IEEE TRANSACTIONS ON BIG DATA,2025,PP(99).
APA	Wu, Zihao.,Zhang, Lu.,Cao, Chao.,Yu, Xiaowei.,Liu, Zhengliang.,...&Liu, Tianming.(2025).Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task.IEEE TRANSACTIONS ON BIG DATA,PP(99).
MLA	Wu, Zihao,et al."Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Task".IEEE TRANSACTIONS ON BIG DATA PP.99(2025).