Vision-based sensory and reinforcement biofeedback system

Abstract

The primary objective of this project is the development of a vision-based sensory and reinforcement system to provide feedback (visual, auditory or tactile) to individuals having complex learning disabilities (PCLD) or Autism Spectrum Disorder (ASD). The feedback will serve as a form of biofeedback, encouraging the individuals to repeat their movements. Such repetition is expected to trigger a learning process by which the individual is able to gain awareness and control over physical action. A detailed study on various object detection algorithms, classification techniques and tracking methods is carried out. Two object detection techniques are selected and implemented, namely the template matching algorithm and the colour detection algorithm. The former technique detects the salient regions in the scene, which consists of an inclined hollow semi-circular gutter, a ball and a skittle, while the latter approach detects the moving objects of interest on the basis of their colour. The location of the ball is computed by dividing the setup into four sections detected by the template matching technique, which are then used to specify the ranges required for detecting the ball's position. The results achieved when testing the system under various circumstances conclude that the goal of the dissertation is met since audible feedback, indicating the ball's location can be provided successfully in the majority of the cases. However the system has some limitations including the fact that varying lighting conditions affect the threshold ranges, which were used to define the colours of the ball and the skittle. Also, the template images are not scale or rotation invariant, hence any slight size or orientation variation might cause issues when attempting to locate the template image on the frame. It is then verified that the system can be implemented in real-time such that in the future, it may be developed and deployed at San Miguel Primary School, so that individuals with intellectual disabilities, who are often unaware of their own movements or lack of movements, are provided with an opportunity to engage with external objects through voluntary activity.