Uncertainty-Aware Hierarchical Reinforcement Learning Robust to Noisy Observations

This work proposes UA-HRL, an uncertainty-aware hierarchical reinforcement learning framework for mitigating the problems caused by noisy sensor data. The system is composed of an ensemble of predictive models that learns the environment's underlying dynamics and estimates the uncertainty through their prediction variances and a two-level Hierarchical Reinforcement Learning agent that integrates the uncertainty estimates into the decision-making process. It is also shown how frame-stacking can be combined with the uncertainty estimation for the agent to make better decisions despite the aleatoric noise present in the observations. In the end, results obtained in a simulation environment are presented and discussed.