The WSSUS pulse design problem in multicarrier transmission

Optimal link adaption to the scattering function of wide-sense stationary uncorrelated scattering (WSSUS) mobile communication channels is still an unsolved problem despite its importance for next-generation system design. In multicarrier transmission, such link adaption is performed by pulse shaping, i.e., by properly adjusting the transmit and receive filters. For example, pulse-shaped offset-quadrature amplitude modulation (OQAM) systems have recently been shown to have superior performance over standard cyclic prefix orthogonal frequency-division multiplexing (OFDM) (while operating at higher spectral efficiency). In this paper, we establish a general mathematical framework for joint transmitter and receiver pulse shape optimization for so-called Weyl-Heisenberg or Gabor signaling with respect to the scattering function of the WSSUS channel. In our framework, the pulse shape optimization problem is translated to an optimization problem over trace class operators which, in turn, is related to fidelity optimization in quantum information processing. By convexity relaxation, the problem is shown to be equivalent to a convex constraint quasi-convex maximization problem thereby revealing the nonconvex nature of the overall WSSUS pulse design problem. We present several iterative algorithms for optimization providing applicable results even for large-scale problem constellations. We show that with transmitter-side knowledge of the channel statistics a gain of 3-6 dB in signal-to-interference-and-noise-ratio (SINR) can be expected.