Optimized Training for Convolutional Neural Network Using Enhanced Grey Wolf Optimization Algorithm
DOI:
https://doi.org/10.31449/inf.v45i5.3497Abstract
Convolutional Neural Networks (CNNs) are widely used in image classification tasks and have achieved significant performance. They have different applications with great success, especially in medical fields. The choice of architecture and hyperparameter settings of the CNN, highly influences both accuracy and its convergence speed. The empirical design and optimization of a new CNN architecture require a lot of expertise and can be very time-consuming. This paper proposes an enhanced Grey Wolf Optimization (GWO) algorithm to efficiently explore a defined space of potentially suitable CNN architectures, and simultaneously optimize their hyperparameters. Moreover, we introduce a spatial resolution reduction for a given image processing task, while taking the skin cancer detection as practical application. Through conducted experiments, we have shown that the obtained results are better than other classification methods in terms of accuracy and convergence speed.References
Mirjalili, S. M., Lewis, A. Grey wolf optimizer. Advances in Engineering Software, 69, (2014). , 46–61.
G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Comput., vol. 18, no. 7, pp. 1527–1554, 2006.
Ali Bakhshi, Nasimul Noman, Zhiyong Chen, Fast Automatic Optimization of CNN Architectures for Image Classification Using Genetic Algorithm
C. Letellier, Chaos in nature, World Scientific Publishing Company, 81 (2013)
K. Price, R. Storn , Differential evolution: a simple evolution strategy for fast optimization. Dr Dobb’s J Software Tools 1997;22(4):18–24.
Coelho Ld, Ayala HV, Mariani VC. A self-adaptive chaotic differential evolution algorithm using gamma distribution for unconstrained global optimization. Appl Math Comput 2014;234:452–9.
U. Yüzgeç, M. Eser, Chaotic based differential evolution algorithm for optimization of baker’s yeast drying process, Egyptian Informatics Journal 19 (2018) 151–163
M. Suganuma, S. Shirakawa, and T. Nagao, “A genetic programming approach to designing convolutional neural network architectures,” in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 497–504, ACM, 2017.
B. Wang, Y. Sun, B. Xue, and M. Zhang, “Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification,” in 2018 IEEE Congress on Evolutionary Computation, CEC 2018 - Proceedings, 2018.
K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, “What is the best multi- stage architecture for object recognition?,” in Proceedings of the IEEE International Conference on Computer Vision, 2009.
T. Yamasaki, T. Honma, and K. Aizawa, “Efficient Optimization of Convolutional Neural Networks Using Particle Swarm Optimization,” in Proceedings - 2017 IEEE 3rd International Conference on Multimedia Big Data, BigMM 2017, 2017.
J. Kennedy, R. Eberhart, "Particle swarm optimization", Proc. IEEE International Conf. on Neural Networks (Perth Australia), 1995.
S. Kumar, D. Datta, S. Kumar Singh, A. T. Azar, and S. Vaidyanathan, “Black hole algorithm and its applications,” Stud. Comput. Intell., 2015.
H. J. Kelley, “Gradient Theory of Optimal Flight Paths,” ARS J., vol. 30, no. 10, pp. 947–954, 1960.
M. Liu, J. Shi, Z. Li, C. Li, J. Zhu, and S. Liu, “Towards Better Analysis of Deep Convolutional Neural Networks,” IEEE Trans. Vis. Comput. Graph., 2017.
J. Yosinski, J. Clune, A. Nguyen, T. Fuchs, and H. Lipson, “Understanding neural networks through deep visualization,” arXiv Prepr. arXiv1506.06579, 2015.
H. A. Ahmed, M. F. Zolkipli, and M. Ahmad, “A novel efficient substitution- box design based on firefly algorithm and discrete chaotic map,” Neural Computing and Applications, 2018.
A. A. Alzaidi, M. Ahmad, H. S. Ahmed, and E. Al Solami, “Sine-Cosine Optimization-Based Bijective Substitution-Boxes Construction Using Enhanced Dynamics of Chaotic Map,” Complexity, vol. 2018, 2018.
A. M. Taha, S.-D. Chen, and A. Mustapha, “Bat Algorithm Based Hybrid Filter- Wrapper Approach,” Adv. Oper. Res., vol. 2015, 2015.
A. M. Taha, S.-D. Chen, and A. Mustapha, “Natural Extensions: Bat Algorithm with Memory.,” J. Theor. Appl. Inf. Technol., vol. 79, no. 1, pp. 1–9, 2015.
Ogudo, K.A.; Muwawa Jean Nestor, D.; Ibrahim Khalaf, O.; Daei Kasmaei, H. A Device Performance and Data Analytics Concept for Smartphones‟ IoT Services and Machine-Type Communication in Cellular Networks. Symmetry 2019, 11, 593.
S. Q. Salih, A. A. Alsewari, B. Al- Khateeb, and M. F. Zolkipli, “Novel Multi-swarm Approach for Balancing Exploration and Exploitation in Particle Swarm Optimization,” in Recent Trends in Data Science and Soft Computing, 2019, pp. 196–206.
Z. A. Al Sudani, S. Q. Salih, Z. M. Yaseen, and others, “Development of Multivariate Adaptive Regression Spline Integrated with Differential Evolution Model for Streamflow Simulation,” J. Hydrol., pp. 1–15, 2019.
A. P. Piotrowski, J. J. Napiorkowski, and P. M. Rowinski, “How novel is the novel black hole optimization approach?,” Inf. Sci. (Ny)., 2014.
L. M. R. Rere, M. I. Fanany, and A. M. Arymurthy, “Simulated Annealing Algorithm for Deep Learning,” in Procedia Computer Science, 2015.
A. R. Syulistyo, D. M. J. Purnomo, M. F. Rachmadi, and A. Wibowo, “Particle swarm optimization (PSO) for training optimization on convolutional neural network (CNN),” J. Ilmu Komput. dan Inf., vol. 9, no. 1, pp. 52–58, 2016.
Boukaye Boubacar Traore , Bernard Kamsu-Foguem , Fana Tangara, Deep convolution neural network for image recognition, Ecological Informatics, November 2018
I. Arel, D. C. Rose, T. P. Karnowski, et al., “Deep machine learning new frontier in artificial intelligence research,” IEEE Computational Intelligence Magazine, vol. 5, no. 4, pp. 13–18, 2010.
M. Matsugu, K. Mori, Y. Mitari, and Y. Kaneda, “Subject independent facial expression recognition with robust face detection using a convolutional neural network,” Neural Networks, vol. 16, no. 5-6, pp. 555–559, 2003.
I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning, vol. 1. MIT press Cambridge, 2016.
W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, “Asurvey of deep neural network architectures and their applications,” Neurocomputing, vol. 234, pp. 11–26, 2017.
Tatsuki Serizawaa, and Hamido Fujita, Optimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization, Computer Science, ArXiv
Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
