This article investigates the potential of Large Language Models (LLMs) in mutation testing, a critical domain where generating effective mutations for testing purposes has been a significant challenge. The authors conducted an empirical study using four LLMs on 440 real bugs, revealing that LLMs produced mutations that not only exhibited greater diversity but also better alignment with real bugs. As a result, these models achieved approximately an 18% increase in fault detection rates compared to traditional methods. The study also examines various factors affecting mutation generation, such as prompt engineering and experiment settings, highlighting their implications on results.
Our study reveals that Large Language Models (LLMs) can significantly improve the generation of mutations used in testing, enhancing fault detection by 18%.
This paper explores how LLMs provide diverse and behaviorally relevant mutations, offering greater utility in mutation testing compared to existing methods.
Collection
[
|
...
]