• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Konferenzschrift
  4. Sparsity in Deep Neural Networks - An Empirical Investigation with TensorQuant
 
  • Details
  • Full
Options
2019
Conference Paper
Title

Sparsity in Deep Neural Networks - An Empirical Investigation with TensorQuant

Abstract
Deep learning is finding its way into the embedded world with applications such as autonomous driving, smart sensors and augmented reality. However, the computation of deep neural networks is demanding in energy, compute power and memory. Various approaches have been investigated to reduce the necessary resources, one of which is to leverage the sparsity occurring in deep neural networks due to the high levels of redundancy in the network parameters. It has been shown that sparsity can be promoted specifically and the achieved sparsity can be very high. But in many cases the methods are evaluated on rather small topologies. It is not clear if the results transfer onto deeper topologies. In this paper, the TensorQuant toolbox has been extended to offer a platform to investigate sparsity, especially in deeper models. Several practical relevant topologies for varying classification problem sizes are investigated to show the differences in sparsity for activations, weights and gradients.
Author(s)
Loroch, Dominik Marek
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM  
Pfreundt, Franz-Josef  
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM  
Wehn, Norbert
Technische Universität Kaiserslautern
Keuper, Janis
IMLA, Hochschule Offenburg
Mainwork
ECML PKDD 2018 Workshops  
Conference
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD) 2018  
Workshop "Decentralized Machine Learning on the Edge" (DMLE) 2018  
Workshop on IoT Large Scale Machine Learning from Data Streams (IOTStream) 2018  
DOI
10.1007/978-3-030-14880-5_1
Language
English
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM  
  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024