上海科技大学知识管理系统

Automated test techniques usually generate unit tests with higher code coverage than manual tests. However, the readability of automated tests is crucial for code comprehension and maintenance. The readability of unit tests involves many aspects. In this paper, we focus on test inputs. The central limitation of existing studies on input readability is that they focus on test codes alone without taking the tested source codes into consideration, making them either ignore different source codes’ different readability requirements or require manual efforts to write readable inputs. However, we observe that the source codes specify the contexts that test inputs must satisfy. Based on such observation, we introduce the Context Consistency Criterion (a.k.a, C3), which is a readability measurement tool that leverages Large Language Models to extract primitive-type (including string-type) parameters’ readability contexts from the source codes and checks whether test inputs are consistent with those contexts. We have also proposed EvoSuiteC3. It leverages C3’s extracted contexts to help EvoSuite generate readable test inputs. We have evaluated C3’s performance on 409 JAVA classes and compared manual and automated tests’ readability under C3 measurement. The results are two-fold. First, The Precision, Recall, and F1-Score of C3’s mined readability contexts are 84.4%, 83%, and 83.7%, respectively. Second, under C3’s measurement, the string-type input readability scores of EvoSuiteC3, ChatUniTest (an LLM-based test generation tool), manual tests, and two traditional tools (EvoSuite and Randoop) are 90%, 83%, 68%, 8%, and 8%, showing the traditional tools’ inability in generating readable string-type inputs. We have conducted a survey based on the questionnaires collected from 30 programmers with varied backgrounds. The results reveal that when C3 identifies readable differences between tests, programmers tend to give similar opinions of the test’s readability of C3.