Investigating bird sound recognition for scientific surveying and citizen science on a smart phone

Abstract

As mobile technology becomes more sophisticated, its use to observe and survey the environment is becoming an integral part in the studies of natural ecosystems. Sensors required to perform certain fieldwork are now available within one compact and relatively inexpensive device. This study addresses the possibility of using mobile-technology to solve the problem of automatically recognizing bird species from the sound they make. Communication by sound plays a central role in the work of people who study birds. Unfortunately, recognizing birds only from their sound is not an easy task and a lot of training is required. In our research we managed to exploit the advantages of the Smart-Phone environment to create a proof of concept, which we named Tringa, that performs the full bird sound recognition lifecycle. The implementation of such a system was built using evolutionary prototyping, structured around the main stages of the process, namely: Sound Capture, Automatic Segmentation, Feature Extraction and Classification. Prior to the actual implementation, a thorough investigation of any related work and technology was conducted. This was mainly done so as the challenges brought by both the complexity of sound recognition and also by the limitations of Smart-Mobile device technology were not underestimated. One important obstacle to overcome was the varying quality of sound signals captured by the omni-directional microphones of smartphones in the uncontrolled recording environments in which birds are typically found. Such approach allowed us to perform a proper evaluation process of our implementation, which in turn revealed very encouraging and satisfactory results. This evaluation process was also key in determining which technology to use and which approaches to take. For the species set we chose for our study; segmenting recordings within the Energy-Time domain and using a k-Nearest Neighbour classifier trained with Mel-frequency Cepstral Coefficients provided the best results. Such work provides a solid foundation for future research in this area where mobile technology can help citizen science projects involving bird studies to increase their efficiency and reliability.