FaceTagger framework : face detection and recognition for videos & images

Abstract

Face tagging is the process of annotating the content of media files with identification information based on facial recognition. Until now, this has largely been a manual process, as computerised aids were somewhat limited in scope and capability. Automated tools give a clear benefit to specific areas such as social networking sites and security systems - users would only need to verify and confirm the automatically suggested results. As a corollary to this, searching for people in annotated videos and images becomes more easily accessible and more accurate. The availability of a framework that handles the entire face tagging process would enable developers to focus on developing their client application. The framework could be easily interfaced from any application, with consistent results guaranteed across the different clients. The project is logically divided into the three main phases required for a fully-functional proof-of-concept prototype: The first phase is the process of face detection - an essential component which ultimately determines the overall accuracy of the tool. The Viola-Jones method was employed to provide a fast and robust way to achieve good results when detecting the presence of faces in an image. This algorithm uses Haar features, integral imaging, and a cascade of AdaBoost classifiers to ascertain whether a region in an image is likely to contain a face or not. After finding a face in an image, the face is tracked in subsequent frames. This is a better way to follow the same face as it eliminates the problem of undetectable tilted faces that is the plague of most face detectors. This method is also more likely to correctly handle the presence of more than one face in the same image or frame. The CamShift algorithm - short for Continuously Adaptive Mean Shift algorithm - was used for face tracking. After obtaining the histogram of the hue values from the detected face, it calculates, for each incoming frame, the probability of a face possibly lying on each pixel in the image/frame. The area being tracked is then moved towards the largest concentration of pixels with the highest probability within a "gravity boundary" based on the previous area tracked. The Eigenface algorithm was used to tackle the biggest hurdle in the process: face recognition against a database of known faces. This algorithm was established in the early 90s but still forms the basis of most face recognition tools being developed today. It takes the principle component analysis of facial images and uses line fitting to try and find the best fitting line that creates the most distance between different faces, thus making it easier to distinguish between faces and be in a better position to recognise another instance of a 'familiar' face (i.e. one that belongs to the trained collection of known faces). If a certain distance threshold is not satisfied for a face under analysis, that face is labelled as unknown. A socket-based server application making the basic face detection/recognition functions available was created. It can be fed videos and images and, after processing the media based on the client application's instructions, transmits the results back to the caller. This makes it possible for various applications to delegate all face recognition operations to this server application. Although the results obtained were deemed to be quite satisfactory on the whole, the project itself proved to be ripe with potential, with more than ample room to grow. Among the limitations noted, one could mention. i) faces that moved too close to each other frustratingly brought the tracking process to a halt; ii) some false positives were produced by the face detector, and; iii) different lighting conditions adversely affected the accuracy and consistency of the face recognition results. Nevertheless, the aim of developing an interoperable framework for use with applications developed in different programming languages was fulfilled. Future work can be directed towards optimising each of the algorithms used to obtain more accurate results. The use of a common database to store a common set of face information between different applications would also be another potential area for improvement and further research.