Conducting a choral ensemble with visually impaired singers – how can research in HCI help?

Abstract

This research project aims to identify how HCI techniques could be used to aid visually impaired singers who wish to partake in choral activities. A low-cost solution that addresses the communication barrier between a conductor and a visually impaired singer is proposed. This makes use of a gesture recognition device and is capable of recognizing the hand gestures of a conductor, interpreting the signal that these gestures communicate and finally transmit the intended signal to a wireless device as audio or haptic feedback. The proposed solution is designed and developed following an iterative prototyping methodology, and evaluated with experts and users related to the domain. Initially, interviews with choral conductors were performed to extract knowledge on the area, as well as challenges that need to be addressed. Literature on existing assistive technologies and HCI solutions for impaired musicians, gesture recognition devices, and real-time communication channels and protocols is reviewed. Evaluations were carried out with experts as well as blindfolded singers who assessed the whole system iteratively. This work provides knowledge on how HCI technology can be of help to this segment in society. It proposes a basis that aids in ameliorating the situation and that focuses on the inclusion of visually impaired people in social activities.