ELROB and EURATHLON: Improving search & rescue robotics through real-world robot competitions

It is a difficult task to compare methods in the field of outdoor robotics. Robotic competitions have become a common means of evaluating the performance of robotic techniques. EURATHLON and ELROB are examples for such outdoor robotics competitions aiming to assess the capabilities of robotic systems in realistic disaster response scenarios. Obviously, this requires special efforts in the metrics and benchmarking domain. Additionally, a major aim of ELROB and EURATHLON is standardization, for example regarding technical interfaces, protocols, data formats; or even complex structures like payload specifications and exchangeable datasets. This paper describes the EURATHLON and ELROB projects, the conducted competitions and, especially, the efforts made in the fields of standardization and benchmarking.