Please use this identifier to cite or link to this item:
https://www.um.edu.mt/library/oar/handle/123456789/113505
Title: | The myth of reproducibility : a review of event tracking evaluations on Twitter |
Authors: | Mamo, Nicholas Azzopardi, Joel Layfield, Colin |
Keywords: | Artificial intelligence Twitterbots Machine learning Data collection platforms Information retrieval |
Issue Date: | 2023-04-05 |
Publisher: | Frontiers Research Foundation |
Citation: | Mamo, N., Azzopardi, J. & Layfield, C. (2023). The myth of reproducibility: A review of event tracking evaluations on Twitter. Frontiers in Big Data, 6, 1067335. |
Abstract: | Event tracking literature based on Twitter does not have a state-of-the-art. What it does have is a plethora of manual evaluation methodologies and inventive automatic alternatives: incomparable and irreproducible studies incongruous with the idea of a state-of-the-art. Many researchers blame Twitter's data sharing policy for the lack of common datasets and a universal ground truth–for the lack of reproducibility–but many other issues stem from the conscious decisions of those same researchers. In this paper, we present the most comprehensive review yet on event tracking literature's evaluations on Twitter. We explore the challenges of manual experiments, the insufficiencies of automatic analyses and the misguided notions on reproducibility. Crucially, we discredit the widely-held belief that reusing tweet datasets could induce reproducibility. We reveal how tweet datasets self-sanitize over time; how spam and noise become unavailable at much higher rates than legitimate content, rendering downloaded datasets incomparable with the original. Nevertheless, we argue that Twitter's policy can be a hindrance without being an insurmountable barrier, and propose how the research community can make its evaluations more reproducible. A state-of-the-art remains attainable for event tracking research. |
URI: | https://www.um.edu.mt/library/oar/handle/123456789/113505 |
Appears in Collections: | Scholarly Works - FacICTAI |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Frontiers_Mamo2023.pdf | 360.94 kB | Adobe PDF | View/Open |
Items in OAR@UM are protected by copyright, with all rights reserved, unless otherwise indicated.