Ali, MazenMazenAliNouy, AnthonyAnthonyNouy2022-05-062022-05-062021https://publica.fraunhofer.de/handle/publica/41553810.1137/20M1360657We consider approximation rates of sparsely connected deep rectified linear unit (ReLU) and rectified power unit (RePU) neural networks for functions in Besov spaces $Balpha_{q}(Lp)$ in arbitrary dimension $d$, on general domains. We show that deep rectifier networks with a fixed activation function attain optimal or near to optimal approximation rates for functions in the Besov space $Balpha_{\tau}(Ltau)$ on the critical embedding line $1/\tau=\alpha/d+1/p$ for arbitrary smoothness order $\alpha>0$. Using interpolation theory, this implies that the entire range of smoothness classes at or above the critical line is (near to) optimally approximated by deep ReLU/RePU networks.enReLU neural networksapproximation spacesBesov spacesdirect embeddingsdirect (Jackson) inequalities003519006journal article