Differential evolution for neural networks learning enhancement

Abstract

Evolutionary computation is the name given to a collection of algorithms based on the evolution of a population toward a solution of a certain problem. These algorithms can be used successfully in many applications requiring the optimization of a certain multi-dimensional function. These algorithms have widely been used to optimize the learning mechanism of classifiers, particularly on Artificial Neural Network (ANN) Classifier. Major disadvantages of ANN classifier are due to its sluggish convergence and always being trapped at the local minima. To overcome this problem, Differential Evolution (DE) has been used to determine optimal value for ANN parameters such as learning rate and momentum rate and also for weight optimization. In ANN, there are many elements need to be considered, and these include the number of input nodes, hidden nodes, output nodes, learning rate, momentum rate, bias parameter, minimum error and activation/transfer functions. These elements will affect the speed of neural network learning. DE has been applied successfully to improve ANN learning from previous studies. However, there are still some issues on DE approach such as longer training time to produce the output and the usage of complex functions in selection, crossover and mutation calculation. In this study, DE is chosen and applied to feed forward neural network to enhance the learning process and the network learning is validated in terms of convergence rate and classification accuracy. Three programs have developed; Differential Evolution Neural Network (DENN), Genetic Algorithm Neural Network (GANN) and Particle Swarm Optimization with Neural Network (PSONN) to probe the impact of these methods on ANN learning using various datasets. The results have revealed that DENN has given quite promising results in terms of convergence rate and smaller errors compared to PSONN and GANN.