Publica
Hier finden Sie wissenschaftliche Publikationen aus den FraunhoferInstituten. Graph kernels and Gaussian processes for relational reinforcement learning
 Machine learning 64 (2006), Nr.13, S.91119 ISSN: 08856125 

 Englisch 
 Zeitschriftenaufsatz 
 Fraunhofer IAIS () 
Abstract
RRL is a relational reinforcement learning system based on Qlearning in relational stateaction spaces. It aims to enable agents to learn how to act in an environment that has no natural representation as a tuple of constants. For relational reinforcement learning, the learning algorithm used to approximate the mapping between stateaction pairs and their so called Q(uality)value has to be very reliable, and it has to be able to handle the relational representation of stateaction pairs. In this paper we investigate the use of Gaussian processes to approximate the Qvalues of stateaction pairs. In order to employ Gaussian processes in a relational setting we propose graph kernels as a covariance function between stateaction pairs. The standard prediction mechanism for Gaussian processes requires a matrix inversion which can become unstable when the kernel matrix has low rank. These instabilities can be avoided by employing QRfactorization. This leads to better and more stable performance of the algorithm and a more efficient incremental update mechanism. Experiments conducted in the blocks world and with the Tetris game show that Gaussian processes with graph kernels can compete with, and often improve on, regression trees and instance based regression as a generalization algorithm for RRL.