A Novel Multi-label Classification Based on PCA and ML-KNN



 

A Novel Multi-label Classification Based on PCA and ML-KNN

 

Di Wu, Dapeng Zhang, Fengqin Yang, Xu Zhou and Tieli Sun​​*​​

School of Computer Science and Information Technology

Northeast Normal University

Changchun, 130117, P. R. China

[e-mail]

 

ReceivedDecember 2010; accepted February 2011

 

Abstract.Multi-label Classification problems are omnipresent.ML-KNN is a multi-label lazy learning approach. The feature of high dimensionsand redundancy of the dataset is not considered by ML-KNN, so the classificationresult is hard to be improved further. Principal Component Analysis (PCA) is apopular and powerful technique for feature extraction and dimensionalityreduction. In this paper, a novel multi-label classification algorithm based onPCA and ML-KNN (named PCA-ML-KNN) is proposed. Experiments on two benchmarkdatasets for multi-label learning show that, PCA processes the dataset in anoptimized manner, eliminating the need of huge dataset for ML-KNN, andPCA-ML-KNN achieves better performance than ML-KNN.

Keywords:Multi-label classification, ML-KNN, Dimension reduction,Feature extraction, Principal Component Analysis (PCA)

 

1.Introduction.Multi-label classification is arousing more and more attention and is increasingly required by many applications in widefields, such as protein function classification, music categorization and semantic scene classification. During the past decade, several multi-label learning algorithms have been proposed, like the multi-label decision tree based learning algorithm [1,2] , the support vector machine based multi-labellearning algorithm [3], the ML-KNN algorithm [4,5], etc.. ML-KNN is derived from the traditional K-nearest neighbor (KNN) algorithm and is presented by Zhang and others. Several empirical studies demonstrated that the dataset for Multi-label classification is bulky, and has the characteristic of high dimensions and redundancy. These features pose a serious obstac1e to any attempt to extract pertinent information, thus make it difficult to improve the multi-label classification algorithms.

PCA is a technique of data analysis [6]. In fact it is a mathematical procedure that uses an orthogonal transformation to convert a set of observations of possibly correlated variables into a set of values of uncorrelated variables called principal components. The most important application of PCA isto simplify the original data. PCA can effectively identify the most important elements in the dataset, eliminate noise and redundancy. Another advantage ofPCA is that it has no parameter restrictions, and can be applied to variousfields.

In this paper, a novel multi-label classification algorithm based on PCA and ML-KNN is proposed for improving the classification performance. PCA is adopted to reduce dataset dimensionality and noise. This isthe first procedure for the classification. Then ML-KNN method is used for rest processing. To verify the effectiveness of PCA-ML-KNN, two datasets, e.g. Sceneand Enron are used, and the experiments report excellent performance.

......

---

Corresponding author