News sentiment effect on financial market trends

Abstract

This project aims to investigate the impact of news articles on financial market trends, such as the Standard & Poor’s 500 index. The main hypothesis of this project is that news articles contain specific characteristics, such as sentiment and events, that can be leveraged to predict the direction of the stock market. The ultimate goal is to develop a Machine Learning (ML) model that can effectively use news article features, alongside other stock market data if needed, to forecast market trends. Market trend is defined as the tendency of financial markets to move in a particular direction over a given time frame. The direction is determined by a number of variables, such as events, speculation, supply and demand and government policy. Predicting these variables reliably is difficult. However, in reality the market is made up of many investors and traders who are willing to buy or sell shares of a specific stock. The common phenomenon here is that investors and traders use news sources to try to predict where the stock price will be in the future. After this evaluation they place their order accordingly. The premise here is that certain events are expected to lead to a positive outlook – hence attracting more buyers – while other events lead to a negative outlook leading to an increase in selling. This change in supply and demand results in a change in the trend of the markets’ value. For individual investors, reading and analysing daily news articles is a time consuming and tedious task. However, Artificial Intelligence (AI) systems can observe patterns over historical data and generalise efficiently. In this project different ML models are developed to analyse the movement of the stock market in relation to daily news articles and forecast the direction for the immediate movements. The business-related articles are obtained using the New York Times (NYT) Application Programming Interface (API). Textual features from news reports are extracted and weighted using different Natural Language Processing (NLP) methods. In essence, these methods should result in an accurate and machine-readable representation of the textual features. The ML models are trained on these features and stock market data to predict the direction, ‘Upwards’ or ‘Downwards’, of a given stock market equity for a given time window, primarily from market open till market close. The possibility of predicting stock market prices has been a subject of debate among researchers, with some stating that it is infeasible due to the market’s volatility and unpredictability. However, the rise of AI solutions has led many to challenge this claim, and this project’s results demonstrate the potential of using news articles as features to forecast the direction of the stock market over a long period. Moreover, the results offer valuable insights into the factors that influence the stock market. All in all, this research showcases the use of AI in forecasting financial market trends.