Automatic recognition of historical buildings in Valletta using smartphone technology

Abstract

Building classification is a widely researched area in computer vision. In this research project a smartphone application is developed which uses computer vision techniques. The idea is introduced by pointing out several advantages and disadvantages of such a system. The application allows the user to take a picture of historical buildings, and then it automatically classifies it and gives information about the building. The technical terms that are used in this project, are described in the following chapter, and previous work from the literature is highlighted. Building recognition is ultimately an object recognition problem, so in the second chapter we look at different approaches to object recognition algorithms applied to various building datasets. An overview of the system is described next, and the design of how the system is implemented is highlighted. The section describes how the user can upload a photo, receive information regarding the respective building and give back feedback to the system. In the same chapter there is also a description of the process to create three distinct datasets using data from thirteen Maltese buildings. Furthermore, an "Unknown" category was included to make the project more realistic. The implementation is discussed in the fourth chapter, and the system to recognise a building is described in detail. The full operation is discussed in detail, i.e. uploading the image, recognising the building, sending back the data to the user, and the user sends back feedback to the system. A flowchart is also drawn to show the full process to recognise the building in a query image. Next, several experiments were implemented, such as using different image processing techniques, or applying the algorithm on various datasets. The application is also distributed to users for evaluation. Results show that the proposed system is very effective. The algorithm gained also an accuracy which is equal to the state-of-the-art on the Zurich Building benchmark dataset. Finally, ideas are discussed to further improve the application, such as implementing augmented reality, and using deep learning algorithms.