Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

On Chow-Liu forest based regularization of deep belief networks

: Sarishvili, A.; Wirsen, A.; Jirstrand, M.


Tetko, I.V.:
Artificial Neural Networks and Machine Learning - ICANN 2019. Workshop and Special Sessions. Proceedings : 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17-19, 2019
Cham: Springer Nature, 2019 (Lecture Notes in Computer Science 11731)
ISBN: 978-3-030-30492-8 (Print)
ISBN: 978-3-030-30493-5 (Online)
International Conference on Artificial Neural Networks (ICANN) <28, 2019, Munich>
Fraunhofer ITWM ()
probabilistic graphs; Restricted Boltzman machines; Chow-Liu trees

In this paper we introduce a methodology for the simple integration of almost-independence information on the visible (input) variables of the restricted Boltzmann machines (RBM) into the weight decay regularization of the contrastive divergence and stochastic gradient descent algorithm. After identifying almost independent clusters of the input coordinates by Chow-Liu tree and forest estimation, the RBM regularization strategy is constructed. We show an example of a sparse two hidden layer Deep Belief Net (DBN) applied on the MNIST data classification problem. The performance is quantified by estimating misclassification rate and measure of manifold disentanglement. Approach is benchmarked to the full model.