Multimodal communication in virtual environments

Abstract

This paper describes research work carried out at the Center for Sprogteknologi (CST) in Copenhagen to develop the multimodal communication components of a 3D virtual world application. This work was part of an interdisciplinary Danish research project – the Staging project – the purpose of which has been to investigate the nature and application potential of 3D virtual worlds, and their impact in theatre and entertainment (see Qvortrup (2000) for an overview). One of the many research activities the project has dealt with, was the design and development of a virtual farm scenario and the exploration of this scenario from a number of different perspectives. Relevant issues were for example the usefulness of theatrical metaphors in 3D applications, the interaction modes possible in them, and last but not least how the user can communicate with the virtual agents. The interaction patterns defined for the agents in the scenario – a farmer and a number of animals – are based on an interactive improvisation defined for the EU-funded Puppet project, which has had an ongoing cooperation with Staging (Klasen et al. 2000). In our work, we have focused on the interaction between the agents and the user, who can communicate his intentions to the farmer and in that way influence the events on the farm