Code profiling for typical scenario testing

Abstract

The process of testing all possible inputs and preconditions is becoming more unfeasible for systems as they are increasing in complexity. In this dissertation we propose a framework which amalgamates testing and machine learning to test libraries by using typical usage execution paths. A typical usage execution path of a library can be expressed as a sequence or a regular grammar. We use Aspect Oriented Programming (AOP) to log the method calls from a library at run time. Clustering techniques such as Complete Link and Single Link are used to extract all the important features from a large set of execution logs A subset of all the logs is used as the input to the two state merging algorithms, Evidence Driven State Merging (EDSM), proposed by Rodney Price and Alergia by Rafael C.Carraco and Jose Oncina. These two algorithms output deterministic finite state automata (DF A) and deterministic stochastic finite state automata (DSF A) which describe the typical usage of the library in different notations. The automata also cover other typical usages which are not part of the sample of the input execution traces. The general automaton induced is tested in a test bench which asserts the effectiveness of the pattern by using black box tests and random testing. This dissertation uses SMTP and SSH to test two open source libraries, DotNetOpenMail and SharpSSH The results obtained show that machine learning and testing can be used hand in hand to provide a testing environment to a library; A library is profiled using execution traces obtained by its users to test the typical ordering of method call routines. The final pattern induced reflects the typical execution traces and other atypical execution traces. Thus other execution traces are covered while testing for a specific usage of the library, providing an attractive alternative to brute force testing. More over the automata obtained can guide future users to develop correct routines from the APL.