Seimetz, M.M.SeimetzNoelle, M.M.NoellePatzak, E.E.Patzak2022-03-032022-03-032007https://publica.fraunhofer.de/handle/publica/21339210.1109/JLT.2007.896810Increased interest in novel modulation formats for optical transmission has come up in recent years. Receivers with interferometrit direct detection can be used to detect arbitrary modulation formats with differentially encoded phases as differential phase-shift keying (DPSK) and differentially phase-encoded star-shaped quadrature amplitude modulation (Star QAM). In this paper, two novel 16-ary modulation formats, which are the 16DPSK and the Star 16QAM (ASK-81)PSK), are characterized for optical transmission for the first time. To be able to identify clear performance tendencies for high-order optical modulation, the novel formats are compared to a wide range of already investigated formats by conducting comprehensive calculations in a uniform simulation environment. The influence of different transmitter structures and decision schemes is considered, and all the systems are characterized with respect to the optical signal-to-noise ratio requirements, dispersion tolerance, and self-phasemodulation (SPM) performance for nonreturn-to-zero (NRZ) and RZ pulse shapes. Moreover, an inherent problem of Star QAM transmission concerning SPM is illustrated, and compensation techniques are examined. The results give a substantial insight into the properties of high-order optical modulation formats.en621535journal article