Video indexing and searching using MPEG-7

Abstract

Due to the exponential increase in audiovisual production and storage, an urgent need is created for fast, efficient and robust video search engines. The number of people tuned into YouTube's videos is clear evidence to how important correct management of audiovisual data is. This led to the development of efficient processing tools that are able to create descriptions of audiovisual material and support the identification or retrieval of audiovisual documents. Besides the significance of such tools, the need for interoperability between devices is also crucial for a reliable video search engine. MPEG-7, also called "Multimedia Content Description Interface", standardizes the description of multimedia content supporting a wide range of applications. Focus is given to four visual descriptors that are proposed in the MPEG-7 standard. These include two colour descriptors: the Colour Layout Descriptor (CLD) and Colour Structure Descriptor (CSD) together with the Homogeneous Texture Descriptor (HTD) and Edge Histogram Descriptor (EHD) used as texture descriptors. Prior to any additions and modifications, the Video Indexer framework indexed video files using any of the aforementioned visual descriptors. The indexed videos were saved in VIX files. These files could then be read and a search could be carried out in them. The main objective of this project was to convert the VIX files to MPEG-7 standard using XML. Supplementary modifications were made to the framework to allow reading from and searching through the MPEG-7 XML files. The search is based on one of the four visual descriptors. An image of the user's choice can be compared to one of these files, and by means of distance functions, the frame shot with the smallest distance to the image can be located. This frame shot will resemble the selected image the most when compared by the visual descriptor defined in the search criteria. In practical terms, this function can be used to search for a particular frame shot in a video file. To further increase functionality, additions were made to the framework to allow a search through multiple videos. This will save time when a large collection of videos is available and a search for a frame shot has to be made through all of them.