Investigating technological challenges involved when adopting smart mobile technology for gesture communication

Abstract

Nowadays, smart mobile devices are continuously gaining momentum, becoming increasingly popular and more sophisticated. The growth of smart mobile technology is creating room for new opportunities involving technological communication. This study addresses the challenges of gestures in mobile devices. Gestures are used for natural communication to convey messages and also as means of expression, therefore being of great value in terms of communication. This research integrates these two means of communication (smart mobile technology and gesture communication) through gesture recognition. While addressing technological challenges related to gesture recognition, these are evaluated in the light of hardware limitations posed by smart mobile devices. By adopting the necessary methodology, technologies and concepts and by investigating the technological challenges within gesture recognition for smart mobile devices, a series of prototypes were designed and developed. These led to the creation of our proof of concept - GiM - Gestures in Mobile, a smart mobile device application which portrays the full gesture recognition lifecycle. The creation of this proof of concept involved a number of stages within the gesture recognition lifecycle, mainly: Image Processing, Feature Extraction and Feature Classification. A number of approaches together with the necessary technologies were comparatively analysed at each stage in order to choose, adopt and integrate the most suitable set of techniques such as Vector-Quantisation and Hidden Markov Models, and tools such as Android SDK, Open CV 4Android and Jahmm to finally create this smart mobile device application. Testing and evaluation were carried out in real-world scenarios, leading to the achievement of satisfactory results. This study is also a foundation for future research in the area of sign language recognition, where gestures can be used as means to overcome communication barriers.