Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

Deep Learning based Affective Sensing with Remote Photoplethysmography

: Luguev, T.; Seuß, D.; Garbas, J.-U.


Institute of Electrical and Electronics Engineers -IEEE-:
54th Annual Conference on Information Sciences and Systems, CISS 2020 : 18-20 March 2020, Princeton, NJ, USA, cancelled due to the COVID-19 (Coronavirus)
Piscataway, NJ: IEEE, 2020
ISBN: 978-1-7281-4084-1
ISBN: 978-1-7281-4085-8
ISBN: 978-1-7281-8831-7
Annual Conference on Information Sciences and Systems (CISS) <54, 2020, Online>
Conference Paper
Fraunhofer IIS ()

Recent studies show that heart rate variability (HRV) is an important physiological characteristic that reflects physiological and affective states of a person. Advancements in the field of remote camera-based photoplethysmography has made possible measurement of cardiac signals using just the raw face videos. Most of existing studies of camera-based cardiovascular monitoring focus on just heart rate (HR) estimation, leaving more interesting case of remote HRV estimation out of scope. However, knowing only the average HR is not enough for affective sensing applications, and measurement of HRV is beneficial. We propose a new framework, which uses deep spatiotemporal networks for contactless HRV measurements from raw facial videos. The proposed framework employs data augmentation technique. It was evaluated on two multimodal databases that consists face videos with synchronized physiological signals. Experiments demonstrate the advantage of our deep learning based approach for HRV estimation. We also achieved promising results for inclusion remote HRV estimation in affective sensing applications.