Enhancement of particle swarm optimization in Elman recurrent network with bounded Vmax function

Abstract

As the widespread modus operandi in real applications, Backpropagation(BP) in Recurrent Neural Networks (RNN) is computationally more powerful than standard feedforward neural networks. In principle, RNN can implement almost any arbitrary sequential behavior. However, there are many drawbacks in BP network, for instance, confinement in finding local minimum and may get stuck at regions of a search space or trap in local minima. To solve these problems, various optimization techniques such as Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) have been executed to improve ANN performance. In this study, we exploit errors optimization of Elman Recurrent Network with Particle Swarm Optimization (ERNPSO) to probe the performance of both networks with bounded Vmax function. Main characteristics of Vmax function are to control the global exploration of particles in PSO. The results show that ERNPSO with bounded Vmax of hyperbolic tangent furnishes promising outcomes in terms of classification accuracy and convergence rate compared to bounded Vmax sigmoid function and standard Vmax function.