Backpropagation is Sensitive to Initial Conditions
John F. Kolen
Jordan B. Pollack
Laboratory for Artificial Intelligence Research,
Computer and Information Science Department,
The Ohio State University, Columbus, OH 43210, USA
Abstract
This paper explores the effect of initial weight selection on feed-forward networks learning simple functions with the backpropagation technique. We first demonstrate, through the use of Monte Carlo techniques, that the magnitude of the initial condition vector (in weight space) is a very significant parameter in convergence time variability. In order to further understand this result, additional deterministic experiments were performed. The results of these experiments demonstrate the extreme sensitivity of backpropagation to initial weight configuration.
