Multi-Task Deep Learning Approach for Contactless Simultaneous Heart Rate and Oxygen Saturation Estimation

Pulse oximeters are commonly used for health monitoring to measure heart rate (HR) and peripheral oxygen saturation (SpO<inf>2</inf>). For better hygiene and comfort, camera based non-contact vital signs monitoring has become increasingly popular. In this work, we present and validate for the first time a novel approach using a multi-task neural network to simultaneously estimate the absolute values of HR and SpO<inf>2</inf> from multispectral video. The approach begins by preprocessing the multispectral video across the spatial, temporal, and frequency domains. The proposed network structure incorporates partially shared feature extractors and task-specific branches, along with a feature map channel attention mechanism and task weight balancing mechanism. This design enhances the extraction of shared shallow features for HR and SpO<inf>2</inf>, leveraging their interrelationships, and then focuses on the more relevant and informative features for each task. The performance is validated through short measurements of 23 healthy participants using a 3D multispectral multimodal cameras system in a breath-holding study approved by the Ethics Committee of the Faculty of Medicine, University of Duisburg-Essen (approval no. 21-10312-BO). In the leave-oneparticipant-out validation scenario, compared with parallel commercial pulse oximeter recordings, the mean absolute error of HR and SpO<inf>2</inf> measurements are 2.97 beats per minute and 2.34 % respectively. And the Pearson correlation coefficient of HR and SpO<inf>2</inf> measurements are 0.54 and 0.68 respectively. The result demonstrates good precision and notable agreement with the commercial pulse oximeter, indicating the potential of this approach to contribute a hygienic and comfortable alternative for simultaneous HR and SpO<inf>2</inf> measurement.