PSO with crossover operator applied to feature selection problem in classification
Abstract
In recent years, there is a large number of features in datasets used in pattern classification, which include relevant, irrelevant, and redundant features. However, irrelevant and redundant features decrease the computational time and reduce the classification performance. Feature selection is a preprocessing technique that which to choose a sub-set of relevant features to achieve a similar or even better classification performance than using all features. This paper presents two new hybrid algorithms for a feature selection called particle swarm optimization with crossover operator (denoted as PSOCO1 and PSOCO2); the algorithms are based on the integration of a particle swarm optimization (PSO) and a crossover operator (CO) of the genetic algorithms. A new relevant features vector (RFV) is introduced and used by our algorithms for execute a crossover operator between the RFV and other features vectors. To demonstrate the effectiveness of these algorithms, we compared them with standard PSO [14], PSO4-2 [8] and HGAPSO [28] on twelve benchmark datasets. The results show that the two proposed algorithms significantly reduce the number of selected features and achieve similar or even better classification accuracy in almost all cases.References
Enny I S, Sri H, Agus H, Retantyo W, Mudjosemedi M (2015) Feature Selection of the Combination of Porous Trabecular with Anthropometric Features for Osteoporosis Screening. International Journal of Electrical and Computer Engineering (IJECE) 5(1): 78--83
Zheng H, Zhang Y (2008) Feature selection for high dimensional data in astronomy. Advances of Space Research 41:1960–1964
Wang Q, Liu F, Wang X (2014) Multi-objective optimization of machining parameters considering energy consumption. International Journal of Advanced Manufacturing Technology 71:1133–1142
Hlaing T (2012) Feature Selection and Fuzzy Decision Tree for Network Intrusion Detection. International Journal of Informatics and Communication Technology 1(2):109--118
Kohavi R, John GH (1997) Wrappers for feature subset selection. Artificial Intelligence 97(1--2):273–324
Colak S, Isik C (2003) Feature subset selection for blood pressure classification using orthogonal forward selection. In: Proceedings of IEEE 29th Annual Bioengineering Conference, pp 122–123
Cotter SF, Kreutz-Delgado K., Rao BD (2001) Backward sequential elimination for sparse vector selection. Signal Processing 81:1849–1864
Xue B, Zhang M, Browne WN (2014) Particle swarm optimization for feature selection in classification: Novel initialization and updating mechanisms. Applied soft computing 18:261–276
Kanan HR, Faez K (2008) An improved feature selection method based on ant colony optimization (ACO) evaluated on face recognition system. Applied Mathematics and Computation 205:716–725
Ani AA (2005) Ant colony optimization for feature subset selection. proceedings of world academy of science, engineering and technology 4:35--38
Gao HH, Yang HH, Wang XY (2005) Ant colony optimization based network intrusion feature selection and detection. Proceedings of the 4th international conference on machine learning and cyberneting. Vol. 6, , pp 3871–3875
Huang CL, Wang CJ (2006) A GA-based feature selection and parameters optimization for suppert vector machine. Expert systems with applications 31:231–240
Shuxin Z, Bin H (2013) Hybrid Feature Selection Based on Improved Genetic Algorithm. TELKOMNIKA Indonesian Journal of Electrical Engineering 11(4):1725--1730
Kennedy J, Eberhart R (1997) A discrete binary version of the particle swarm algorithm. In: IEEE International Conference on Systems, Man, and Cybernetics. Vol. 5, pp 4104–4108
Unler A, Murat A (2010) A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3):528–539
Yang CS, Chuang LY, Ke CH, Yang CH (2008) Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation (CEC’08), pp 2093–2098
Lin SW, Lee ZJ, Chen SC, Tseng TY (2008) Parameter determination of support vector machine and feature selection using simulated annealing approach, Applied soft computing 8:1505–1512
Gherboudj A, Chikhi S (2011) BPSO Algorithms for Knapsack Problem. Communications in Computer and Information Science. 162:217–227
Chuang LY, Tsai SW, Yang CH (2011) Improved binary particle swarm optimization using catfish effect for feature selection. Expert system with application 38(10):12699–12707
Yang CS, Chuang LY, Ke CH (2008) Boolean binary particle swarm optimization for feature selection. In: IEEE Congress on Evolutionary Computation, pp 2093–2098
Chuang LY, Chang HW, Tu CJ, Yang CH (2008) Improved binary PSO for feature selection using gene expression data. Computational Biology and Chemistry 32(29):29–38
Liu YN, Wang G, Chen HL, Dong H (2011) An Improved Particle Swarm Optimization for Feature Selection. Journal of Bionic Engineering 8(2):191–200
Fdhila R, Hamdani T, Alimi A (2011) Distributed MOPSO with a new population sub-division technique for the feature selection. In: IEEE 5th International Symposium on Computational Intelligence and Intelligence Informatics, pp 81–86
Hongfeng W, Dingwei W, Shengxiang Y (2007) Triggered Memory-Based Swarm Optimization in Dynamic Environments. In: Workshops on Applications of Evolutionary, pp 637–646
Sun J, Feng B, Xu W (2004) Particle Swarm Optimization with Particles Having Quantum Behavior. In: Congress on Evolutionary Computation, Portland, USA, pp 325–331
Hamed HNA, Kasabov NK, Shamsuddin (2011) SM Quantum inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks. Evolutionary algorithms 1:133–148
Mohemmed A, Zhang M, Johnston M (2009) Particle swarm optimization based AdaBoost for face detection. In: IEEE Congress on Evolutionary Computation, pp 2494–2501
Ghamissi P, Benediktsson JA (2015) Feature selection based on hybridization of genetic algorithm and particle swarm optimization. IEEE Geoscience and remote sensing letters 12(2):309–313
Frank A, Asuncion A, UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA, 2016. <http://archive.ics.uci.edu/ml>
Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2008) The WEKA Data Mining Software: An Update. ACM SIGKDD Explorations Newsletter 11:10–18.
Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
