Using objects and scene attributes to estimate depth from single monocular images : relative depth estimation between two objects using semantic features

Abstract

The depth of objects plays an integral part of several computer vision tasks. Using just a single image it is very difficult to obtain an accurate depth level without matching the scene with a Three-Dimensional (3D) representation of the image using any additional feature information of the scene. The state of the art systems in this area are complex Neural Network models trained on stereo image data to predict depth per pixel. Fortunately, in some tasks, only the relative depth between objects is required. This dissertation moves away from other approaches that make use of multiple images or scene reconstruction and explores the use of semantic information in the learning process of machine learning models to aid the learning and prediction of relative depth. The problem tackled in this dissertation is addressed using a classification approach where geometrical features based on the object bounding boxes, object labels and scene attributes are computed and extracted and used as inputs to pattern recognition machine learning models models to predict relative depth between the objects (i.e object is behind, in-front or objects are at equal depth). We evaluate the solution with the monodepth Convolutional Neural Network (CNN) trained on the Cityscapes dataset, and note a positive boost of 14% accuracy over the results reported by the monodepth depth prediction CNN. This result demonstrates that data driven depth estimation models are capable of producing results that are superior to other complex depth prediction mechanisms.