A novel ensemble of classifiers for microarray data classification

Abstract

Micorarray data are often extremely asymmetric in dimensionality, such as thousands or even tens of thousands of genes and a few hundreds of samples. Such extreme asymmetry between the dimensionality of genes and samples presents several challenges to conventional clustering and classification methods. In this paper, a novel ensemble method is proposed. Firstly, in order to extract useful features and reduce dimensionality, different feature selection methods such as correlation analysis, Fisher-ratio is used to form different feature subsets. Then a pool of candidate base classifiers is generated to learn the subsets which are re-sampling from the different feature subsets with PSO (Particle Swarm Optimization) algorithm. At last, appropriate classifiers are selected to construct the classification committee using EDAs (Estimation of Distribution Algorithms). Experiments show that the proposed method produces the best recognition rates on four benchmark databases.

Introduction

Microarray technology has provided the ability to measure the expression levels of thousands of genes simultaneously in a single experiment. Each spot on a microarray chip contains the clone of a gene from a tissue sample. Some mRNA samples are labelled with two different kinds of dyes, for example, Cy5 (red) and Cy3 (blue). After mRNA interact with the genes, i.e., hybridization, the color of each spot on the chip will change. The resulted image reflects the characteristics of the tissue at the molecular level [1].

In recent years, research has showed that accurate cancer diagnosis can be achieved by performing microarray data classification. Various intelligent methods have been applied in this area. But the microarray data consists of a few hundreds of samples and thousands or even ten thousands of genes. It is extremely difficult to work in such a high dimension space using traditional classification methods directly. So gene selection methods have been proposed and developed to reduce the dimensionality. These include principal components analysis (PCA) [8], Fisher-ratio, t-test, and correlation analysis. Along with the feature selection methods, intelligent methods have been applied for microarray classification, such as support vector machine (SVM) [7], K nearest neighbor (KNN) [6], artificial neural network (ANN) [1]. But high accurate classification is difficult to achieve. Most intelligent classifiers are apt to be over-fitted. Recent years, ensemble approaches have been proposed. It combines multiple classifiers together as a committee to make more appropriate decisions for classifying microarray data instances. It offers improved accuracy and reliability. Much research has shown that a sufficient and necessary condition the approach outperforms its individual members is that the base classifiers should be accurate and diverse. An accurate classifier is one that has an error rate of better than randomly guessing classes for new instances, and two classifiers are diverse if they make different errors on common data instances [9]. So there are two important aspects to be focused on ensemble approaches.

First aspect is how to generate diverse base classifiers. In traditional, re-sampling has been widely used to generating training datasets for base classifiers learning. This method is much too random and due to the small numbers of samples, the datasets may be greatly similar. In this paper, different methods such as correlation analysis, Fisher-ratio are firstly applied to generate feature subsets. Since variety of selection methods, some features selected in the subsets are different with each other and all of them are informative genes. Then re-sampling the feature subsets to form learning datasets. Owing to the datasets forming from different feature subsets, it may be much more various and more efficient.

The second aspect is how to combine the base classifiers. In this paper, an intelligent approach for constructing ensemble classifiers is proposed. The methods first training the base classifiers with particle swarm optimization (PSO) algorithm, and then select the appropriate classifiers to construct a high performance classification committee with estimation of distribution (EDA) algorithm. Experiment show that the proposed methods produce the best recognition rates.

The paper is organized as follows. The feature selection methods are introduced in Section 2. The particle swarm optimization used to train the neural networks which are employed as the base classifiers is depicts in Section 3. The optimal design method for constructing ensemble classifiers is described in Section 4. Section 5 gives the simulation results. Finally, we present some concluding remarks.