Improving neural networks training using experiment design approach

Abstract

This project involves the use of Neural Networks (NN) for function approximation. Conventionally, the parameters of a neural network are tuned by minimizing an objective function based on a pre-determined set of training data. This training paradigm is passive in the sense that the neural network only learns from the training patterns presented to it by the environment or a teacher. It may be more useful if the neural network could 'actively' obtain the training samples itself sequentially by interacting with its environment. There are several methods of selecting training data from input space for neural networks which include D-optimal and Max-min design approaches. Consider a function approximation problem (Neural Network using Radial Basic Function structure) and limit the amount of training data, say (m) from N amount of possible data. Randomly select the m data set for conventional training algorithm. One more data (m+ 1) is entered to train the NN again. This data is selected by two methods: random and Experiment Design Approach (D-optimal and Maxmin Distance). The performances of the two approaches are then compared. It was found that the NN trained using the data obtained using experiment design approaches approximated the unknown function better than the NN that is trained when the data are selected randomly. Maxmin Distance Approach is independent of the NN model while Doptimal point is dependent on the NN model used.