Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

SPSA for layer-wise training of deep networks

: Wulff, B.; Schuecker, J.; Bauckhage, C.


Kůrková, V.:
Artificial Neural Networks and Machine Learning - ICANN 2018. Proceedings, Part III : 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018
Cham: Springer International Publishing, 2018 (Lecture Notes in Computer Science 11141)
ISBN: 978-3-030-01424-7
ISBN: 978-3-030-01423-0
ISBN: 978-3-030-01425-4
International Conference on Artificial Neural Networks (ICANN) <27, 2018, Rhodes>
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
01/S18038C; ML2R
Conference Paper
Fraunhofer IAIS ()

Concerned with neural learning without backpropagation, we investigate variants of the simultaneous perturbation stochastic approximation (SPSA) algorithm. Experimental results suggest that these allow for the successful training of deep feed-forward neural networks using forward passes only. In particular, we find that SPSA-based algorithms which update network parameters in a layer-wise manner are superior to variants which update all weights simultaneously.