Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Neural Network Compression via Learnable Wavelet Transforms

: Wolter, Moritz; Lin, Shaohui; Yao, Angela


Farkaš, I. ; European Neural Network Society:
Artificial Neural Networks and Machine Learning - ICANN 2020. Proceedings. Pt.II : 29th International Conference on Artificial Neural Networks, Bratislava, Slovakia, September 15-18, 2020
Cham: Springer Nature, 2020 (Lecture Notes in Computer Science 12397)
ISBN: 978-3-030-61615-1 (Print)
ISBN: 978-3-030-61616-8 (Online)
International Conference on Artificial Neural Networks (ICANN) <29, 2020, Online>
Fraunhofer SCAI ()
Wavelets; network compression

Wavelets are well known for data compression, yet have rarely been applied to the compression of neural networks. This paper shows how the fast wavelet transform can be used to compress linear layers in neural networks. Linear layers still occupy a significant portion of the parameters in recurrent neural networks (RNNs). Through our method, we can learn both the wavelet bases and corresponding coefficients to efficiently represent the linear layers of RNNs. Our wavelet compressed RNNs have significantly fewer parameters yet still perform competitively with the state-of-the-art on synthetic and real-world RNN benchmarks (Source code is available at Wavelet optimization adds basis flexibility, without large numbers of extra weights.