Challenges Using FPGA Clusters for Distributed CNN Training

While FPGAs are well-established in the field of CNN inference, there is a lack of research regarding CNN training. To mitigate this, we develop an easy-to-use framework for CNN training on network-attached FPGA clusters. Tightly pipelined layer parallelism promises to facilitate a high utilization across a whole cluster. This comes with challenges, which we discuss in this paper.