The Effects of Model Compression on the Robustness of Deep Neural Networks

Deep neural networks have made significant progress in recent years. However, these improvements are closely tied to an increased number of parameters and resources required for inference. Model compression techniques, such as pruning and quantization, demonstrated that it is possible to reduce the number of parameters and memory requirements respectively while maintaining a high test set accuracy. In this work, we investigate the effects of model compression on the robustness of neural networks by iteratively pruning and quantizing neural networks with different precisions across multiple architectures and two datasets. We identify a set of metrics and methods and provide a framework to facilitate the evaluation of the robustness of compressed neural networks. We find that the mean test set accuracy hides a considerable amount of detail in how different classes are impacted disproportionally by model compression. Furthermore, we qualitatively show that compressed models are functionally different from their original counterparts using saliency maps. Given the widespread deployment of compressed neural networks on mobile devices, it is crucial to evaluate and understand the disparate impact of compression on specific samples and classes.