Cheating at testing : a gaming metaphor for system testing

Abstract

Testing has become an important aspect in the development of a software system, as quality has to be maintained. Most of today's large systems exhibit a high level of complexity, which tends to make the testing process more difficult. Furthermore, changes are continuously done to a developing system, which in tum have to be tested as well. The idea of "cheating at testing" is conceptualized and investigated in this dissertation. As a gamer uses cheats in order to be able to complete some level in a game, a tester is guided through the testing process by being shown the areas, within an application that uses the Swing toolkit, that exhibit higher levels of risk. Therefore, once the changes are identified, a "relative code churn measure" is used to calculate the risky areas based on the additions and modifications done to a piece of code. The risky areas are then analyzed, with the help of a call graph, to determine the components they make use of and if they do, the latter will be highlighted in the generated heatmap. The evaluation presented here shows that the designed approach did reflect the high risk areas in the system's user interface. Thus if in some revision, a line in one method has been modified, and a new feature was implemented, then the components being used by the new feature are highlighted in a much stronger colour. In addition, the calculations of the high risk areas based on the number of changes done per file, and the number of changes done per method were analyzed, with the results given by the latter being identified as better. The evaluation results also indicated that the ability to consider the risk areas within a number of revisions helps during the testing process.