Refined Pruning Techniques for Feed-forward Neural Networks
Anthony N. Burkitt
Computer Sciences Laboratory, Research School of Physical Sciences,
Australian National University, GPO Box 4, Canberra, ACT 2601, Australia
Peer Ueberholz
Physics Department, University of Wuppertal,
Gauss-Strasse 20, D-5600 Wuppertal 1, Germany
Abstract
Pruning algorithms for feed-forward neural networks typically have the undesirable side effect of interfering with the learning procedure. The network reduction algorithm presented in this paper is implemented by considering only directions in weight space that are orthogonal to those required by the learning algorithm. In this way, the network reduction algorithm chooses a minimal network from among the set of networks with constant E-function values. It thus avoids introducing any inconsistency with learning by explicitly using the redundancy inherent in an oversize network. The method is tested on boolean problems and shown to be very useful in practice.
