Opportunities and applications of ultrasound sensing on unmodified consumer-grade smartphones

A person's smartphone is a cornucopia of information. Be it personal data extracted from contacts and calendar entries or the current location via GPS. The huge variety of sensors in today's mobile phones makes these devices a prime target for human activity recognition. The smartphone is no longer solely seen as actuator in smart environments, enabling the user to control auxiliary devices and sensors, but can now play a vital part in the network of sensing information itself. Especially in the area of human activity recognition, camera-based or body-worn systems are predominant. While they achieve high accuracy, these methods often suffer from privacy issues or obtrusiveness and consequently social stigma. In this thesis, I present an unobtrusive approach to perceive the vicinity surrounding the phone by leveraging the properties of ultrasound sensing. The device emits ultrasonic waves via its speaker and records the echo via the microphone. By analyzing the received signal, I can deduct certain movements, e.g. gestures performed above the phone, but also more complex motions involving the whole body of the user. I outline various experiments to estimate the feasibility of ultrasound sensing in different scenarios as well as propose an algorithm and mobile application that can classify given gestures and activities performed by the user. The system is able to recognize predefined gestures with an overall accuracy of 81% over six different users and can detect human activities up to 2m away.