An overview of techniques aimed at automatically generating oracles from tests

Abstract

While significant testing is carried out throughout software development lifecycles, given the size and complexity of modern software systems, the possibility of post-deployment bugs remains a real concern. Although users would typically report any encountered bugs themselves, such bugs could tarnish the company’s reputation and expose vulnerabilities within the system which could be exploited, potentially costing the company a fortune. One way of reducing this risk is to include monitors with the running software such that any occurring bugs are immediately detected, reported, and potentially mitigated automatically. A significant problem with such a technique, however, is that existing tools require the software engineers to specify the properties to be checked in a formal language. This typically involves training personnel to use the tool and the language, as well as having to dedicate extra time to write the properties themselves. This adds up to the already expensive development process that software companies are constantly trying to optimize hence deterring the interest in including the use of runtime monitoring. [Excerpt]