Automatic sports match highlight generation

Abstract

Big sporting events are reported in real-time by many news outlets in the form of minute by minute updates. Usually these outlets also produce a news summary of the event published right after the end. This is generally a time consuming task, where the journalists face a race against time to be among the first to publish their report at the end of the match, in order to publish the articles at the time when interest is at its highest. This project is focused on the detection of highlights in sporting events, focusing on football matches. By using both audio and textual features to detect highlights, the system then produces a summary report intended to provide an overview of the game. This report is not meant to replace the role of the journalist in any way, but rather to assist the journalist by creating an output of the commentator speech during detected highlights. The journalist would then make the necessary corrections and if needed, expand on some further points. The solution is split in 4 parts. Textual features and audio features both contain valuable information that could help in identifying highlights as well as identifying what type of highlight it is. Thus, the first 2 parts consists of training two separate classifiers for the audio and text components using a DenseNet architecture and a 1D ConvNet architecture respectively. Following this, an ensemble classifier is then trained to determine the timestamps of any detected highlights based on the confidence and results of the previous 2 classifiers. Finally, the system has the ability to use summarization techniques utilizing Skip-Thought Vectors and K-Means Clustering. These four components together make up the architecture which, by using the commentators’ speech and timestamps, is able to produce a summary report of the highlights of the analysed games. Following the evaluation of each component of the system, there is an indication that the system proposed is effective in performing classification tasks. The ensemble classifier achieved an AUC score of 98.1% and managed to detect all the highlights within the test set. The summarization techniques trained were not used in the final output due to how the input text is provided in the dataset, since the input text is not a list of sentences, rather a list of words, which makes them ineligible for the summarization techniques implemented to be used effectively. The scripts to compile the dataset including the video, audio and closed captions are published to assist further research within this area.