ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation

doi:10.1109/TSE.2024.3382365

	ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation
	Yutian Tang 1; Zhijie Liu2 ; Zhichao Zhou2 ; Xiapu Luo 3
	2024
发表期刊	IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (IF:6.5[JCR-2023],7.0[5-Year])
ISSN	2326-3881
EISSN	1939-3520
卷号	PP 期号:99 页码:1-19
发表状态	已发表
DOI	10.1109/TSE.2024.3382365
摘要	Recent advancements in large language models (LLMs) have demonstrated exceptional success in a wide range of general domain tasks, such as question answering and following instructions. Moreover, LLMs have shown potential in various software engineering applications. In this study, we present a systematic comparison of test suites generated by the ChatGPT LLM and the state-of-the-art SBST tool EvoSuite. Our comparison is based on several critical factors, including correctness, readability, code coverage, and bug detection capability. By highlighting the strengths and weaknesses of LLMs (specifically ChatGPT) in generating unit test cases compared to EvoSuite, this work provides valuable insights into the performance of LLMs in solving software engineering problems. Overall, our findings underscore the potential of LLMs in software engineering and pave the way for further research in this area. IEEE
关键词	ChatGPT Search-based Software Testing Large Language Models
URL	查看原文
收录类别	EI
语种	英语
出版者	Institute of Electrical and Electronics Engineers Inc.
EI入藏号	20241415848930
EI主题词	Application programs
EI分类号	721.1 Computer Theory, Includes Formal Logic, Automata Theory, Switching Theory, Programming Theory ; 723 Computer Software, Data Handling and Applications ; 723.1 Computer Programming ; 723.2 Data Processing and Image Processing ; 723.5 Computer Applications
原始文献类型	Article in Press
来源库	IEEE
引用统计	正在获取...
文献类型	期刊论文
条目标识符	https://kms.shanghaitech.edu.cn/handle/2MSLDSTB/357304
专题	信息科学与技术学院_硕士生信息科学与技术学院_博士生
作者单位	1.University of Glasgow, United Kingdom 2.ShanghaiTech University, China 3.Department of Computing, Hong Kong Polytechnic University, Hong Kong SAR, China
推荐引用方式 GB/T 7714	Yutian Tang,Zhijie Liu,Zhichao Zhou,et al. ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation[J]. IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,2024,PP(99):1-19.
APA	Yutian Tang,Zhijie Liu,Zhichao Zhou,&Xiapu Luo.(2024).ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation.IEEE TRANSACTIONS ON SOFTWARE ENGINEERING,PP(99),1-19.
MLA	Yutian Tang,et al."ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation".IEEE TRANSACTIONS ON SOFTWARE ENGINEERING PP.99(2024):1-19.