Indexing of high-dimensional data in DBMS

Abstract

Novel applications rely on data with a high number of attributes and a much richer set of data types. Furthermore, applications that gather data from sensors generate datasets that include numerous dimensions. It is, therefore, recommended that in such database management systems, indexes are utilized in attempts to retrieve data more efficiently against the cost of maintaining them. Many researchers have endeavoured to create indexing solutions for managing high dimensional data. A staggering number of factors affect indexing on multi-dimensional data. Moreover, single dimension indexing structure techniques do not generally apply in multidimensional databases. This project plans to test whether an index method is worth implementing. Two types of datasets are going to be used; a spatial dataset and a high-dimensional dataset. Different queries will be run on the different datasets, after which the query plan of the query is recorded and analysed. The query planner is configured to run the queries under three different scenarios. The first scenario being with no indexes, a second scenario where the index is forcibly used and finally, with all configurations enabled, where the query planner would be free to choose the execution plan. This evaluation focuses on the differences of the recorded query plans, and calculations are done to see when index is worth implementing. Furthermore, this project aims to further reinforce the belief that indices should be used for high-dimensional data.