Question answering using Wikipedia

Abstract

Unstructured information sources written in natural language, such as Wikipedia, make available huge amounts of factual information. With such information sources and the availability of powerful search engines, a user may easily nd documents relevant to a query. However, retrieving a particular piece of information from these documents may be time consuming. This gave rise to the question answering problem; the process of using computers to nd succinct answers to questions posed in a natural language. In this work, a scalable open-domain question answering system is designed and developed. While this task has been widely studied in the past, we constrain our approach to using Wikipedia, the free encyclopedia, as the exclusive source of information. We present a robust architecture which can be expanded beyond a few types of questions. The focus is mainly on factoid questions and yes/no questions, but other types, such as decision questions, are also studied. While we use only the documents' plain text to answer questions, the implementation allows for other features, which have previously been studied in isolation, to be added. We deal with several sub-tasks in the question answering problem, namely question focus extraction, question classi cation, information retrieval and answer extraction and selection. Results show that the proposed methods achieve adequate performance on simple questions and yes/no questions. In addition, extra e ort put into the question classi cation task, which has received extensive attention in research, contributed to achieving state-ofthe- art performance with room for further improvement.