RJMIM: A New Feature Selection Method Based On Joint Mutual Information

Liwen Peng

Abstract


Feature selection based on information theory plays an important role in classification algorithm due to its computational efficiency and independent from classification method. It is widely used in many application areas like datamining, bioinformatics and machine learning. But drawbacks of these methods are the neglect of the feature interaction and overestimation of features significance due to the limitations of goal functions criterion. To address this problem, we proposed a new feature goal function RJMIM. The method employed joint mutual information and information interaction,which alleviates the shortcomings of overestimation of the feature significance as demonstrated both theoretically and experimentally. The experiments conducted to verify the performance of the proposed method, it compared with four well-known feature selection methods use three publically available datasets from UCI.The average classification accuracy and C4.5 classifier is used to assess the effectiveness of RJMIM method.


Full Text:

PDF

References


Akadi A.E., Ouardighi A.E. (2008). A powerful feature selection approach based on mutual information, International Journal of Computer Science & Network Security, 8(4), 116-121.

Battiti R. (1994). Using mutual information for selecting features in supervised neural net learning, IEEE Transactions on Neural Networks, 5(4), 537-50.

Bennasar M., Hicks Y. (2015). Feature selection using joint mutual information maximization, Expert Systems with Applications, 42(22), 8520–8532.

Bennasar M., Setchi R. (2013). Feature interaction maximization, Pattern Recognition Letters, 34(14), 1630-1635.

Brown G., Pocock A. (2012). Conditional likelihood maximisation: a unifying framework for information theoretic feature selection, Journal of Machine Learning Research, 13(1), 27-66.

Brycki B., Kowalczyk I. (2008). Information-theoretic feature selection in microarray data using variable complementarity, IEEE Journal of Selected Topics in Signal Processing, 2(3), 261-274.

Chen Y.W., Lin C.J. (2006). Feature extraction, foundations and applications, Studies in Fuzziness and Soft ComputingSpringer-Verlag, 89-117.

Cheng H., Qin Z., Feng C., Wang Y., Li F. (2011). Conditional mutual information-based feature selection analyzing for synergy and redundancy, Etri Journal, 33(2), 210-218.

Frohlich H., Chapelle O. (2003). Feature selection for support vector machines by means of genetic algorithm, 2(11), 142-148.

Guyon I., Elisseeff A. (2003). An introduction to variable and feature selection, Journal of machine learning research, 3(6), 1157-1182.

Jakulin A. (2003). Attribute interactions in machine learning, Lecture Notes in Computer Science, 1-15.

Jakulin A.(2005). Machine learning based on attribute interactions (Doctoral dissertation, Univerza v Ljubljani).

Kohavi R., John G.H. (1997). Wrappers for feature subset selection. Artificial Intelligence, 97(1), 273-324.

Kwak N., Choi C.H. (2002). Input feature selection for classification problems. IEEE Transactions on Neural Networks, 13(13), 143-59.

Lan Y.H., Liu C.W., Li Z.S. (2016). A Feature Selection Method for CBIR Mammography CAD, Revista Tecnica De La Facultad De Ingenieria Universidad Del Zulia, 39(1), 282-289

Lewis D.D. (1992). Feature selection and feature extraction for text categorization, In Proceedings of the workshop on Speech and Natural Language, 212-217.

Lin S.W., Ying K.C. (2012). An intelligent algorithm with feature selection and decision rules applied to anomaly intrusion detection, Applied Soft Computing, 12(10), 3285-3290.

McGill W.J. (1954). Multivariate information transmission, Psychometrika, 19(2), 97-116.

Meyer P.E., Bontempi G. (2006). On the use of variable complementarity for feature selection in cancer classification, Lecture Notes in Computer Science, 17(5), 91-102.

Peng H., Long F. (2005). Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy, IEEE Transactions on Pattern Analysis & Machine Intelligence, 27(8), 1226-38.

Swingle B. (2012). Renyi entropy, mutual information, and fluctuation properties of fermi liquids, Physical Review B Condensed Matter, 86(4), 7794-7794.

Vergara J.R., Estévez P.A. (2015). A review of feature selection methods based on mutual information, Neural Computing & Applications, 24(1), 175-186.

Yang H.H., Moody J. (2000). Data visualization and feature selection: new algorithms for nongaussian data, Advances in Neural Information Processing Systems, 12, 687-693.

Zhao Z., Liu H. (2009). Searching for interacting features in subset selection. Intelligent Data Analysis, 13(2), 207-228.

Zhu, Z., Ong, Y.S. (2007). Wrapper–filter feature selection algorithm using a memetic framework. IEEE Transactions on Systems, Man, and Cybernetics, 37(1), 70-76.


Refbacks

  • There are currently no refbacks.


Revista de la Facultad de Ingeniería,

ISSN: 2443-4477; ISSN-L:0798-4065

Edif. del Decanato de la Facultad de Ingeniería,

3º piso, Ciudad Universitaria,

Apartado 50.361, Caracas 1050-A,

Venezuela.

© Universidad Central de Venezuela