Deep Learning Based UAV Payload Recognition

Due to the increased availability of unmanned aerial vehicles (UAVs), the demand for automated counter-UAV systems to protect facilities or areas from misused or threatening UAVs is growing. Fundamental for these systems are fast and accurate detection as well as identification of potential threats to initiate countermeasures. Criteria to classify the potential threat are UAV type and payload. Though thermal or electro optical (EO) imagery have been widely applied for the detection task, other sensor modalities, i.e. acoustic, radar and radio frequency, are predominately used for UAV type and payload classification. In this work, we examine the potential of UAV payload classification in EO imagery, which facilitates direct interpretability by human operators. For this, we compare conventional CNN-based architectures and recent architectures exploiting self-attention mechanisms such as Vision Transformers. The different architectures are trained and evaluated on a novel dataset composed of own recordings of UAVs with and without payload, imagery crawled from the Internet and imagery taken from publicly available UAV datasets.