Automated monitor generation from jUnit tests

Abstract

Testing is an integral part of any software development life-cycle. A frequently used testing method is unit testing, allowing developers to systematically test each component of their system, with each test going into as much detail as the developer deems is necessary. Unit testing forgoes the actual system execution, running the tests in a specially created environment. This is limiting to developers who might require a means of monitoring certain critical points during the system's execution. Runtime verifications allows for the monitoring of system functionality during execution, verifying that the system, in execution, adheres to a set of predefined properties. There is a degree of overlap present between the two forms of verification, with both methods sharing similar types of conditions to define success. Using both methods of testing, however, is a time and resource consuming affair. The proposed solution is a guided but automated monitor generation system, that generates monitors for runtime verification from given jUnit test cases. The system is guided when it comes to deciding on key semantic choices to be taken from the jUnit test case input. This solution would allow developers to use both means of verification, increasing their coverage without much impact on their workload and avoiding redundant coding. The solution was evaluated by means of a case study whereupon a series of monitors were generated from two jUnit tests to properly highlight the different monitor behaviours and test the different functions that could be generated. The semantic values where upheld throughout the process even when erroneous semantics where given during the generation stage. Good, bad and invalid monitor outputs were achieved where expected. The solution implemented still displayed a number of limitations. One particular output served to highlight a design limitation that results in certain cases of generated monitors giving invalid results.