Progressive Feature Extraction with a Greedy Network-growing Algorithm
Ryotaro Kamimura
Electronic mail address: ryo@cc.u-tokai.ac.jp.
Information Science Laboratory and Future Science and Technology Joint Research Center,
Tokai University,
1117 Kitakaname Hiratsuka Kanagawa 259-1292, Japan
Abstract
In this paper, a new information theoretic method called the greedy network-growing algorithm is proposed. The method is called "greedy," because a network with this algorithm grows while absorbing as much information as possible from outside. The method is based upon information theoretic competitive learning and can solve the fundamental problems inherent in competitive learning, such as the dead neurons and inappropriate number of neurons problems. The new model can grow networks by repeatedly maximizing information content and by gradually extracting salient features from input patterns. Because the new model can cope with inappropriate feature detection in the early stage of learning, extracted features should cover most of the input patterns. The new method is applied to political data analysis, medical data analysis, and information science education analysis. Experimental results confirm that the new method can acquire significant information and that more explicit features can be extracted.
