Quantifying Uncertainty in Spiking Neural Networks

Spiking Neural Networks (SNNs) are inspired by the human brain and in contrast to conventional Artificial Neural Networks (ANNs), they operate on spikes. Consequently, they possess high theoretical potential in significantly reducing the computational cost and recent developments allow SNNs to be deployed in power-constrained applications such as robotics. But with this promising outlook, research concerning AI safety also becomes crucial. More specifically, the development of methods for estimating the uncertainty that is captured in a prediction is of high relevance. Deep ensembles have been empirically proven to achieve both improved classification performance and robust estimates of uncertainty that can be decomposed in its different sources. To keep up with conventional ANNs, spiking deep ensembles are a valid option for robust uncertainty estimation. As this approach requires multiple models to run in parallel, it also suffers from high computational and memory overhead which is critical particularly in power-constrained devices. This master thesis adopts the idea of Ensemble Distribution Distillation (EDD), which is a model compression technique and has been already successfully applied to ANNs, and transfers it to the spiking domain. During the process of EDD, a single SNN learns to retain both, the improved classification performance of the ensemble as well as the information about its diversity indicating different forms of uncertainty contained in a prediction. For classification, EDD uses Prior Networks that explicitly parametrize a Dirichlet distribution to learn the implicit output distribution of the ensemble. As a single Dirichlet is very limited and potentially not able to capture the full behavior of the ensemble, we also extend EDD with Prior Networks that parametrize a mixture of Dirichlet distributions. In this work, we study the behavior of SNNs trained via EDD on the four neuromorphic datasets N-MNIST, CIFAR10-DVS, DVS-Gesture, and SL-Animals-DVS and compare them to the original spiking ensemble in terms of classification and calibration performance. Furthermore, we assess the quality of the uncertainty estimates on the tasks of misclassification and Out-of-Distribution (OOD) detection. Additionally, we investigate the influence of the input time window determining the spikes that are used for inference, to produce predictions as soon as possible.