Spectral and temporal multiplexing for multispectral fluorescence and reflectance imaging using two color sensors

Fluorescence imaging can reveal functional, anatomical or pathological features of high interest in medical interventions. We present a novel method to record and display in video rate multispectral color and fluorescence images over the visible and near infrared range. The fast acquisition in multiple channels is achieved through a combination of spectral and temporal multiplexing in a system with two standard color sensors. Accurate color reproduction and high fluorescence unmixing performance are experimentally demonstrated with a prototype system in a challenging imaging scenario. Through spectral simulation and optimization we show that the system is sensitive to all dyes emitting in the visible and near infrared region without changing filters and that the SNR of multiple unmixed components can be kept high if parameters are chosen well. We propose a sensitive per-pixel metric of unmixing quality in a single image based on noise propagation and present a method to visualize the high-dimensional data in a 2D graph, where up to three fluorescent components can be distinguished and segmented.