Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

A neural network based on first principles

: Baggenstoss, P.M.


Institute of Electrical and Electronics Engineers -IEEE-; IEEE Computer Society; IEEE Signal Processing Society:
IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2020. Proceedings : May 4-8, 2020, Barcelona, Spain
Piscataway, NJ: IEEE, 2020
ISBN: 978-1-5090-6631-5
ISBN: 978-1-5090-6632-2
International Conference on Acoustics, Speech and Signal Processing (ICASSP) <45, 2020, Barcelona>
Conference Paper
Fraunhofer FKIE ()

In this paper, a Neural network is derived from first principles, assuming only that each layer begins with a linear dimension-reducing transformation. The approach appeals to the principle of Maximum Entropy (MaxEnt) to find the posterior distribution of the input data of each layer, conditioned on the layer output variables. This posterior has a well-defined mean, the conditional mean estimator, that is calculated using a type of neural network with theoretically-derived activation functions similar to sigmoid, softplus, and relu. This implicitly provides a theoretical justification for their use. A theorem that finds the conditional distribution and conditional mean estimator under the MaxEnt prior is proposed, unifying results for special cases. Combining layers results in an auto-encoder with conventional feed-forward analysis network and a type of linear Bayesian belief network in the reconstruction path.