A study on the context-specific nature of actionable alert identification

Abstract

Automated static code analysis (ASA) tools generate a large quantity of alerts, a significant proportion of which are usually false positives. Actionable alert identification techniques (AAITs) are used to reduce the amount of unactionable alerts through classification and prioritisation of alerts. This is achieved through the use of alert factors such as alert type, priority, cyclomatic complexity and age· of file. There are many indications in the literature which suggest that the success of AAITs lies in making the solution context-specific. The work presented here investigated the validity of this claim in two ways. First an investigation was carried out as to whether or not the selection of alert factors depends on the context. Interviews were conducted with five ICT companies in Malta with different contexts. There are strong indications that each company has its own specific context, which translates into the need of a context-specific AAIT. Other interesting observations from these interviews included that context does not stop at company level, there is a distinction between core code, test code, etc., alert factors can be combined and used in different ways and companies with less stringent deadlines are interested in all kinds of bugs, even code style bugs. As part of the second research question, an AAIT was implemented using an Expert System, designed for the context of the company Ixaris Systems Ltd. The priorities output by the AAIT were compared to a dataset prioritised by the user according to the company's context. The classification results were comparable to previous work, while the prioritisation results would benefit from further improvement. Writing Expert System rules proved to be more complex in this context than initially expected. Prioritisation depended heavily on the bug types, of which there were many (circa 495), further complicated by Ixaris' use of the Spring framework.