An extensible framework for human centred software testing

Abstract

Software testers carry out their day-to-day jobs in a complex and dynamic context. That is to say that, they need to continuously juggle information from a multitude of sources, each of which can change at any given time. They then use this in- formation to carry out a risk-based assessment of which tasks to prioritise at any given time. The sheer volume, variety and velocity of change of this information leads to a phenomenon called Information Anxiety, which has been shown to cause knowledge workers to carry out sub-optimal work. Information anxiety is the stress caused by the inability to access, understand, or make use of information necessary for employees to do their job. In this research it is hypothesised that information anxiety can be reduced by enabling just in time information exchanges of context relevant information with the tester. As a result testers may exhibit an increase in performance due to lower work- load levels - which can materialise across different dimensions, including cognitive but also physical and temporal. This hypothesis informed the research direction, whereby three studies have been devised to (a) investigate and define the tester context, (2) determine how information needs change throughout testing activities and (3) generate insights into how context relevant and just-in-time information may influence the testers performance and overall effectiveness. These studies have been conducted through a mixed-method approach, including a review of literature, a series of participatory studies at specialised tester gatherings in Malta and London as well as through an empirical lab-based study. The latter involved the development of a prototype framework which enabled just-in-time context relevant information exchanges, aimed at generating insights into the effectiveness of the proposed approach. Through multiple modified Delphi method card sorting exercises, the testers' context was defined as the characteristics of the tester, the system under test, the task guided by the test strategy, and the environment. Semi structured interviews with testers uncovered that testers need requirements, acceptance criteria, test cases, and test data amongst other types of information. Furthermore, it was confirmed that testers do in fact experience information anxiety. Finally, it was determined through a case study that enabling just in time exchanges of context relevant in- formation does in fact improve the temporal efficiency of the testers (time required to complete a task) and a reduction in the testers' cognitive workload was ob- served. This work contributes a research-based definition of the testers context, insights into how testers information needs change over time and initial insights into the effect of just-in-time information exchanges on testers performance.