Mutual information based input feature selection for classification problems

Cang, Shuang; Yu, Hongnian

Abstract

The elimination process aims to reduce the size of the input feature set and at the same time to retain the class discriminatory information for classification problems. This paper investigates the approaches to solve classification problems of the feature selection and proposes a new feature selection algorithm using the mutual information (MI) concept in information theory for the classification problems. The proposed algorithm calculates the MI between the combinations of input features and the class instead of the MI between a single input feature and the class for both continuous-valued and discrete-valued features. Three experimental tests are conducted to evaluate the proposed algorithm. Comparison studies of the proposed algorithm with the previously published classification algorithms indicate that the proposed algorithm is robust, stable and efficient.

Citation

Cang, S., & Yu, H. (2012). Mutual information based input feature selection for classification problems. Decision Support Systems, 54(1), 691-698. https://doi.org/10.1016/j.dss.2012.08.014

• Journal Article Type: Article
• Acceptance Date: Aug 17, 2012
• Online Publication Date: Aug 24, 2012
• Publication Date: 2012-12
• Deposit Date: Jun 15, 2022
• Journal: Decision Support Systems
• Print ISSN: 0167-9236
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 54
• Issue: 1
• Pages: 691-698
• DOI: https://doi.org/10.1016/j.dss.2012.08.014
• Keywords: Feature ranking, Optimal feature set, Mutual information, Classification
• Public URL: http://researchrepository.napier.ac.uk/Output/2879337