Topic modelling of newspaper comments using embedding vectors and clustering

Abstract

As the use of the Internet and online social media increases, text data is becoming an ever more important source of data. To this end, a vast number of techniques have been developed in the field of Natural Language Processing. These include n-grams, skip-grams, the Bag-of-Words model, Term Frequency-Inverse Document Frequency, stemming, lemmatisation, embedding vectors, and clustering techniques. This dissertation investigates the theoretical foundations of these techniques, and they are then applied to a dataset of online newspaper comments written between 2008 and 2017 obtained from the Times of Malta website. In particular, the FastText algorithm (Bojanowski et al., 2017) is used to transform each unique word in the dataset to a vector representation known as a word embedding by means of an underlying neural network framework. The word embeddings are then used to obtain clusters by means of the k-means clustering algorithm. Vector representations are also obtained for each online newspaper comment, where again similar comments are assigned similar representations. The obtained representations, which are known as document embeddings, are then also clustered using k-means clustering. The results obtained from the in-depth analysis of the data show that the vast majority of comments are political in nature, with comments related to sports, arts and culture being less frequent than possibly expected. In addition, a number of topics were identified as being more prevalent during some time periods than during others. These include divorce in 2011, as well as Maltese citizenship in 2013 and Russia’s annexation of Crimea in 2014. Furthermore, the morning-after pill and corruption were two topics that were highly discussed in 2016.