• English
  • Deutsch
  • Log In
    Password Login
    Research Outputs
    Fundings & Projects
    Researchers
    Institutes
    Statistics
Repository logo
Fraunhofer-Gesellschaft
  1. Home
  2. Fraunhofer-Gesellschaft
  3. Scopus
  4. Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions
 
  • Details
  • Full
Options
2023
Journal Article
Title

Approximation Theory of Tree Tensor Networks: Tensorized Univariate Functions

Abstract
We study the approximation of univariate functions by combining tensorization of functions with tensor trains (TTs) - a commonly used type of tensor networks (TNs). Lebesgue Lp-spaces in one dimension can be identified with tensor product spaces of arbitrary order through tensorization. We use this tensor product structure to define different approximation tools and corresponding approximation spaces of TTs, associated with different measures of complexity. The approximation tools are shown to have (near to) optimal approximation rates for functions with classical Besov smoothness. We then use classical interpolation theory to show that a scale of interpolated smoothness spaces is continuously embedded into the scale of TT approximation spaces and, vice versa, we show that the TT approximation spaces are, in a sense, much larger than smoothness spaces when the depth of the tensor network is not restricted but are embedded into a scale of interpolated smoothness spaces if one restricts the depth. The results of this work can be seen as both an analysis of the approximation spaces of a type of TNs and a study of the expressivity of a particular type of neural networks (NNs) - namely feed-forward sum-product networks with sparse architecture. We point out interesting parallels to recent results on the expressivity of rectifier networks.
Author(s)
Ali, Mazen
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM  
Nouy, Anthony
Journal
Constructive approximation  
Open Access
DOI
10.1007/s00365-023-09620-w
Language
English
Fraunhofer-Institut für Techno- und Wirtschaftsmathematik ITWM  
Keyword(s)
  • Approximation spaces

  • Besov spaces

  • Direct (Jackson) and inverse (Bernstein) inequalities

  • Matrix product states

  • Neural networks

  • Tensor networks

  • Tensor trains

  • Cookie settings
  • Imprint
  • Privacy policy
  • Api
  • Contact
© 2024