Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Policy learning using SPSA

: Ramamurthy, R.; Bauckhage, C.; Sifa, R.; Wrobel, S.


Kůrková, V.:
Artificial Neural Networks and Machine Learning - ICANN 2018. Proceedings, Part III : 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018
Cham: Springer International Publishing, 2018 (Lecture Notes in Computer Science 11141)
ISBN: 978-3-030-01424-7
ISBN: 978-3-030-01423-0
ISBN: 978-3-030-01425-4
International Conference on Artificial Neural Networks (ICANN) <27, 2018, Rhodes>
Bundesministerium für Bildung und Forschung BMBF (Deutschland)
01/S18038C; ML2R
Conference Paper
Fraunhofer IAIS ()

We analyze the use of simultaneous perturbation stochastic approximation (SPSA), a stochastic optimization technique, for solving reinforcement learning problems. In particular, we consider settings of partial observability and leverage the short-term memory capabilities of echo state networks (ESNs) to learn parameterized control policies. Using SPSA, we propose three different variants to adapt the weight matrices of an ESN to the task at hand. Experimental results on classic control problems with both discrete and continuous action spaces reveal that ESNs trained using SPSA approaches outperform conventional ESNs trained using temporal difference and policy gradient methods.