Method to Determine Most Sensitive and Accurate Mediapipe Face-Mesh Keypoints for Speech Therapy Exercises

Google's Mediapipe face mesh algorithm is the de facto standard for facial keypoint detection from RGB videos due to its high performance and low error. Although its use has shown convincing results for applications that require keypoints around the eyes, mouth, and nose, it falls short for applications that require alternative facial areas due to the limited availability of Mediapipe keypoints.To overcome this challenge, we propose an approach to identify the most suitable keypoints based on two factors: 1) a minimal matching error distance between optimal target coordinates and 2) a maximal intensity of movement variability per keypoint. The suitability of this approach will be evaluated for speech therapy exercises for home-based self-training via a corresponding smartphone app.