Comparing the performance of different approaches to discriminant analysis

Abstract

Discriminant Analysis (DA) is a statistical technique used to classify entities into pre- defined groups. DA consists of two important steps, referred to as the discrimination step and the classification step respectively. Through linear discrimination, a linear boundary which separates the groups is found, by analyzing the group differences. Classification then uses the information described by the discrimination step in order to classify any "new" entities whose group membership is not known. Classical discrimination and classification methods proposed by Sir R. A. Fisher will be presented. However these classical methods fail in the high-dimensional context, where the number of variables p in the dataset is larger than the total number of observations n. In the literature this is referred to as the "large p, small n" problem. This thesis addresses one possible approach to solve this problem, by reducing the high p-dimensional variable space to a lower q-dimensional space, such that q < n. By doing so, classical techniques may then be applied on the reduced data. Dimension reduction techniques construct q new components by taking linear combinations of the original variables, keeping as much of the information present in the original data as possible. Dimension reduction methods based on Principal Components Analysis (PCA) and on Partial Least Squares (PLS) are presented. It is shown that the PLS method results in more accurate classifications than the method using principal components. The performance of these techniques is first tested on simulated data, followed by two applications on real-life data.