Probabilistic Information Capacity of Hopfield Associative Memory
Kesig Lee
Current address: R & D Center, Consumer Electronics Business, Samsung Electronics, 416 Maetan--3 Dong, Kwonsun Gu, Suwon City, Kyungi--Do Korea
S. C. Kothari
Department of Computer Science,
Iowa State University, Ames, Iowa 50011, USA
Dongwan Shin
Current address: University of Suwon, Do Korea
Department of Statistics,
Iowa State University, Ames, Iowa 50011, USA
Abstract
This paper defines a formal probabilistic notion for the information capacity of the Hopfield neural network model of associative memory. A mathematical expression is derived for the number of random binary patterns that can be stored as stable states in a Hopfield model of memory with n neurons with a given probability. The derivation is based on a new approach using two powerful mathematical techniques: Brown's Martingale Central Limit Theorem and Gupta's transformation of the probability integral for a special case of the correlation matrix. The new approach provides a way for rigorously analyzing the complex dynamics of the Hopfield model. Our approach refines the current heuristic methods, which rely on simplifying assumptions about the dynamics of the model.
