Three-term backpropagation algorithm for classification problem

Abstract

Standard Backpropagation Algorithm (BP) is a widely used algorithm in training Neural Network that is proven to be very successful in many diverse application. This algorithm utilizes two term parameters which are Learning Rate, α and Momentum Factor,β. Despite the general success of this algorithm, there are several drawbacks and limitations which some of them are the existence of local minima, slow rates of convergence and some of the modification of BP algorithm requires complex and costly calculations at each iteration, which offset their faster rates of convergence. To overcome this problem, a third learning parameter, Proportional Factor (γ) has been proposed by Zweiri et. al., (2003). This new algorithm is called Three-Term BP. This study investigates the performance of Three-Term BP and compares its performance with standard BP. To achieve this objective, experiments were conducted by implementing Three-Term BP to three dataset which are Balloon, Iris and Cancer dataset. These datasets represents small, medium and large scale data respectively. The results obtained showed that Three-Term BP only outperforms standard BP while using small scale data but not in case of medium and large dataset. This might be caused by the instability of the network while using medium and large dataset as it has been proven in analysis part of the study.