Penalized linear discriminant analysis for high-dimensional data

Abstract

The term discriminant analysis (DA) refers to a class of statistical techniques which are used to solve the problem of assigning an object into one of a number of predefined groups. DA is often divided into the discrimination step and the classification step. The discrimination step is concerned with the descriptive aspect of DA, where a boundary which best separates the groups, is obtained by analyzing the group differences. The classification step is concerned with the predictive aspect of DA, where observations whose group membership is unknown, are allocated to one of the respective groups. This dissertation deals with linear discriminant analysis (LDA), where the boundary separating the groups is linear. The main focus will be on Fisher's interpretation of LDA. It is well-known that LDA is negatively affected when the data is high-dimensional, where the number of features, p, is larger than the number of observations, n. This dissertation explores a particular regularization technique as a possible solution. This technique involves a slight modification to Fisher LDA, coupled up with penalization through the use of the Lasso penalty function, and will be referred to as WT-PLDA-L1. It will be used to extend Fisher LDA to the high-dimensional setting, while also aiming to obtain interpretable outputs, due to the variable selection properties of the Lasso penalty. The performance of this technique will be analyzed on three real-life datasets, and compared to other regularization techniques available in the literature.