Investigating search-based testing techniques for test data generation

Abstract

Software testing has always been the most popular way to gain confidence in the quality of a software system. Although it cannot prove a system bug-free, well-engineered tests give us a high degree of confidence that a system will behave according to how it is expected to. The price of testing is significant, arguably taking half of the resources allocated for a particular project. The need for testing to be less resource-hungry and better is, therefore, of utmost importance. One way to reduce the impact of the cost is to automate test data/case generation. There are several techniques that try to achieve this such as Symbolic Execution, Model-Based test case generation, random testing and Search-Based Software Testing (SBST). SBST is one of the techniques that have been receiving much attention lately due to its natural approach and evident benefits. In this project we investigate the application of SBST techniques to automated unit test generation. This work leads to the development of a modular framework that generates unit test data using an SBST approach. We then use the developed framework to evaluate three search techniques: Hill Climbing, Simulated Annealing and Genetic Algorithm; as well as a fourth custom greedy algorithm. The search algorithms are compared to one another and to a random search algorithm (no learning factor). The results obtained indicate that the greedy approach as well as the genetic algorithm perform generally better than the other techniques. The results obtained are arguably unimpressive when compared to other tools found on the market, but this is mainly due to the limited scope of our project, and this applies to all the search techniques used. The results also show that the learning factor of a search technique is only one of the success factors, alongside performance and efficiency.