Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Reducing the computational effort of optimal process controllers for continuous state spaces by using incremental learning and post-decision state formulations

: Senn, M.; Link, N.; Pollak, J.; Lee, J.H.

Preprint urn:nbn:de:0011-n-2872901 (520 KByte PDF)
MD5 Fingerprint: 72c7e57885d912ca86e4abaea9379e35
Created on: 19.5.2016

Journal of process control 24 (2014), No.3, pp.133143
ISSN: 0959-1524
Deutsche Forschungsgemeinschaft DFG
GRK 1483/A5;
Journal Article, Electronic Publication
Fraunhofer IWM ()
optimal control; continuous state space; dynamic programming; approximate dynamic programming; artificial neural network

Multistage optimization problems that are represented by Markov Decision Processes (MDPs) can be solved by the approach of Dynamic Programming (DP). However, in process control problems involving continuous state spaces, the classical DP formulation leads to computational intractability known as the ‘curse of dimensionality’. This issue can be overcome by the approach of Approximate Dynamic Programming (ADP) using simulation-based sampling in combination with value function approximators replacing the traditional value tables. In this paper, we investigate different approaches of ADP in the context of a deep cup drawing process, which is simulated by a finite element model. In applying ADP to the problem, Artificial Neural Networks (ANNs) are created as global parametric function approximators to represent the value functions as well as the state transitions. For each time step of the finite time horizon, time-indexed function approximations are built. We compare a classical DP approach to a backward ADP approach with batch learning of the ANNs and a forward ADP approach with incremental learning of the ANNs. In the batch learning mode, the ANNs are trained from temporary value tables constructed by exhaustive search backwards in time. In the incremental learning mode, on the other hand, the ANNs are initialized and then improved continually using data obtained by stochastic sampling of the simulation moving forward in time. For both learning modes, we obtain value function approximations with good performance. The cup deep drawing process under consideration is of medium model complexity and therefore allows us to apply all three methods and to perform a comparison with respect to the achieved efficiency and the associated computational effort as well as the decision behavior of the controllers.