Investigate the use of ultrasonic sensor for human pose estimation in smart environments

Human monitoring is a major research direction in computer vision, with application in smart living assistants, human-computer interaction, surveillance, health monitoring, etc. This variety of applications has led to the design of many human monitoring systems in order to extract information about environment inhabitances based on different technologies. In computer vision, this task can be achieved by generating a 2D skeleton representing the human body. However, users do not favor constant camera monitoring. This thesis investigates the use of ultrasonic sensors for human pose estimation, which have a very low cost and require only minimalistic infrastructure. We do this by establishing a framework for data collection of human poses, using an ultrasound sensor and a camera for labeling the data. We collected data from 25 people, performing 4 different activities and evaluated the collected data on 4 different artificial neural network architectures by training them and comparing their performance against each other, showing that an LSTM architecture achieved results up to 67% accuracy. The use of a non-visual input stream for pose estimation is also motivated by the less privacy intrusive nature of ultrasound data, compared with videos of homes and people inside them with the application in smart living environments.