Beigy, Hamid
Coauthors(s): M. R. Meybodi
Amirkabir Univ. of Technology
Computer Engineering Dept.
Computer Engineering Dept. Amirkabir Univ. of Technology Hafez Ave. Tehran, Iran

Adaptation of Parameters of BP Algorithm Using Learning Automata

Backpropagation (BP) algorithm is a systematic method for training multilayer neural networks. Despite of the many successful applications of backpropagation, it has many drawbacks. For complex problems it may require a long time to train the networks, and it may not train at all. Long training time can be the result of the non-optimal parameters. It is not easy to choose appropriate value of the parameters for a particular problem. The parameters are usually determined by trial and error. If the parameters are not chosen appropriately slow convergence, paralysis and continuos instability can result [1-4]. Moreover the best values of the parameters at the beginning of training may not be so good later. In this paper, we incorporate a technique into BP algorithm for adaptation of steepness parameter and momentum factor in order to achieve higher rate of convergence. By interconnection of fixed structure learning automata (FSLA) to the feedforward neural networks, we apply learning automata scheme for adjusting these parameters based on the observation of random response of neural networks. The main motivation in using learning automata as an adaptation algorithm is to use its capability of global optimization when dealing with multi-modal surface. The feasibility of proposed method is shown through simulations on three learning problems: exclusive-or, encoding problem, and digit recognition. These problem are chosen because they have different error surfaces and collectively present an environment that is suitable to determine the effect of proposed method. The simulation results show that the adaptation of these parameters using this method not only increases the convergence rate of learning but it increases the likelihood of escaping from the local minima. Computer simulations provided in this paper indicate that at least a magnitude of savings in running time can be achieved when FSLA is used for adaptation of momentum factor and steepness parameters. Also, simulations indicate that the FSLA approach performs much better than variable structure learning automata (VSLA) approach.